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ABSTRACT 


The Profile Measuring Device (PMD) was developed at the George C Marshall 
Space Flight Center following the loss of the Space Shuttle ChalLengoi It i- 
a rotating gauge used to measure the absolute diameters of mating fe.»-uros of 
redesigned Solid Rocket Motor (SRM) field joints Diameter tolerances or 
these features are typically ±0.005 inches and it is required that the PMD 
absolute measurement uncertainty be within this tolerance. In this analysis 
we find the absolute accuracy of these measurements to be ±0.00375 inches, 
worst case, with a potential accuracy of ±0.0021 inches achievable by improved 
temperature control. 



1. INTRODUCTION 


In this document we present error estimates for the absoLute accuracy 
of part diameters measured with the Profile Measuring Device (PMD), 
developed at the Marshall Space Flight Center (MSFC) . The PMD is used 
to measure the relevant geometry of the mating surfaces of the Space 
Shuttle Solid Rocket Motors (SRMs). This analysis applies specifically 
to PMD Serial Numbers 002 and 003, which are mechanically more robust 
versions of the prototype S/N 001 and which employ an absolute radius 
mastering fixture of simple geometry. A sketch of the PMD system is 
shown in Figure 1 . 

As will be shown, the uncertainties in absolute dimensions determined by 
the PMD are strongly coupled to the various thermal environments 
encountered during the calibration and part measurement processes. By 
international agreement the dimensions of mechanical objects are those 
which exist at a uniform temperature of 68 2 3 F (20°C) Ideally, all 

precision dimensional metrology should be performed at this temperature 
When this is not possible, for reasons of convenience and/or cost, there 
is a price to be paid in measurement uncertainty. At some point, 
thermal errors will equal or exceed the entire working tolerance of the 
part being measured. When this occurs, the only logical choices are to 
improve the thermal environment or to relax the working tolerance?. 

As a guide to understanding the effects of temperature /ariations in 
dimensional metrology, we include as Appendix A a copy of ANSI Standard 
B89 . 6 . 2- ] 973 , "Temperature and Humidity Environment for Dimensional 
Measurement . " 


2 . TECHNICAL APPROA CH 

We have divided the PMD error analysis into a sequence of steps based 
upon the way the device is used in practice. A typical part measurement 
cycle proceeds as follows: 

1) CALIBRATION 

The absolute radius of the PMD radiaL arm is determined h ,* 
comparison with a wave length -compensc ted laser interferometer 
system. This mastering is performed for each of the 
interchangeable measuring tips and occurs in some 
approximately known set of environmental conditions. In 
particular, the average temperature of the radial arm during 
calibration is estimated to be T AC , based upon measurement at 
one radial location. 

2) MEASUREMENT 

SRM tang and/or clevis features are measured. During this 
process, the PMD tips are changed in order to access all of 
the relevant features. These measurements are performed in 
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some (generally different) set of approximately known 

snv i r omne n t a I conditions. Tlie pdi i temperature is measured at 
four circumferential locations, and the average part 

temperature is assigned an estimated value of T PM . The 
average temperature of the radial arm is estimated to be T AM , 
based upon measurement at one radial location. The difference 
T am -T ac Is used to correct the radial data for thermal 
expansion of the radial arm. 

3) ANALYSIS 

The measured set of polar coordinate data pairs (v L , ) is 

reduced by a software analysis routine to yield the part 
dimensions of interest. The routine proceeds by first 
removing the PMD centering error using a least-squares fit to 
a circle. Then, the feature perimeter is estimated by summing 
the chords connecting adjacent points and multiplying the sum 
by a constant. This constant (1.000012693) is the ratio of 
the circumference of a circle to the perimeter of a 360-sided 
inscribed polygon. Finally, the feature diameter is 

calculated by dividing the perimeter by x. 

4) EXTRAPOLATION TO 68 °F 

Part feature dimensions are extrapolated to 68°F using a 
thermal expansion calculation and the temperature difference 
T P m *68°F. 

An examination of this measurement sequence suggests a way to 
decompose the PMD error analysis into a set of plau ibly 
uncorrelated components which can be individually estimated 
and then combined in quadrature. We have chosen the following 
error components : 

1) CALIBRATION ERROR 

Error associated with the mastering process. 

Diameter error = AD cal • 

2) MECHANICAL ERROR 

Error contribution from the geometry and moving 
elements of the PMD. 

Diameter error = • 

3) THERMAL ERROR 

Errors in thermal expansion calculations. 

Diameter error = AD therm . 
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4) SQfTW ARE EB&QE 


ErrOi. introduced in tha analysis algorithms. 

Diameter error = AD S0FT . 

In estimating the contributions to the total PMD measurement 
error from the individual error sources, we are guided by the 
work of R.R. Donaldson, who first developed a systematic 
approach to error budgeting for precision machines (see 
Appendix B) . 

The first step involves estimating the peak- to-valley (PV) 
error amplitudes associated with each individual error source. 
In some instances this can be done by a reasonably 
straightforward calculation. In most cases, however, these 
error amplitudes are plausible estimates guided by experience 
and by comparison with the known behavior of other precision 
machines. The most important requirement in this process is 
to analyze the system at a sufficient level of detail so that 
potentially significant sources of error are not inadvertently 
omitted . 


Once the error amplitudes are assigned, they must be combined 
to yield a composite error using a combinatorial rule. There 
is no known rigorously correct way to do this for mechanical 
systems since the detailed contributions and correlations of 
the individual errors are not known a priori . A worst-case 
estimate can be arrived at by simply adding all PV amplitudes 
together, but this is conservative in the extreme since it is 
highly unlikely that all sources of error will be 
simultaneously at a maximum and in the same direction. 


Another method of error combination, as suggested by 
Donaldson, is to compute a total RMS error from individual RMS 
contributions according to: 


RMS 


TOTAL ~ 



( 1 ) 


where the relati.T 

PV, = K ; RMS t (2) 

relates the RMS amplitudes to the corresponding PV values. 

The constant K depends upon the probability distribution of 
the individual errors which, again, is not generally known. A 
reasonably conservative approacn is to use a uniform 
distribution of the individual errors, in which case 
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K - 273 = 3.46 and Equation (!) becomes 


rws total 


1 

273 



( 3 ) 


Because Equation (3) neglects any possible error correlations, 
there will be a tendency to underestimate the total error. 

In light of these observations, we have adopted an approach 
whicP has recently been used in the Precision Engineering 
Program at Lawrence Livermore National Laboratory in the 
design error budget for a new, high-accuracy measuring machine 
for the Department of Energy. Here, we estimate the composite 
error by averaging the (over-conservative) sum of PV errors 
and the (under-conservative) RrtS result: 


where 




TOTAL 


+ PV 


'total 


PV. 


TOTAL 


4* 

- I 


PV. 




( 4 ) 


( 3 ) 


Lacking any more rigorous theoretical guidance, we believe 
that this approach will yield a plausible estimate of PMD 
absolute accuracy. 


The estimated diameter errors are then calculated for each of 
the four error components: calibration, mechanical, thermal, 
and software. Since we believe these components to be 
uncorrelated, the estimated system error is then calculated by 
quadrature : 

A^total ~ [-CAL)* 4 (AD mech ) 2 + (4D therm ) 2 + (AD soft ) 2 

Equation (6) represents the central result of our estimation 
procedure. We now proceed to develop the detailed error 
components . 
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3. CALIBEAI.LQN £g £0 ,R 


The absolute radius of the PHD is determined using a calibration fixture 
consisting of a laser interferometer system and a precision inspection 
block. The vacuum wavelength of the laser is measured by the 
manufacturer by frequency comparison with an iodine-stabilized He-Ne 
laser. The latter is an internationally recognized transfer standard of 
length which realizes the definition of the meter with an uncertainty of 
approximately 1 part in 10 10 . 

Measurement errors associated with the calibration process consist of 
interferometer errors and mechanical errors. Interferometer errors are 
length -dependent and are calculated assuming a nominal measured 
displacement of 144 inches. 

The vacuum wavelength of the metrology laser is known, by frequency 
comparison, to 1 part in 10 7 or better displacement uncertainty = 0.1 
parts per million (ppm)]. It is necessary to correct the vacuum 

wavelength for the refractive index of air during calibration. This 
correction is primarily determined by air pressure, temperature, and 
relative humidity. Based upon witnessed calibrations and an examination 
of the methods used to determine the environmental variables, we 
estimate the following systematic errors: (a) pressure uncertaintv = 

0.1 in Hg; (b) temperature uncertainty = 2°F; (c) humidity 
uncertainty = 10% RH. The respective error contributions are 1 ppm 

(pressure), 1 ppm (temperature), and 0.1 ppm (humidity). 

The refractive index of air is also affected by changes in composition, 
particularly -hose due to carbon dioxide and organic solvents. We 

conservatively estimate an error of 1 ppm associated with composition 
uncertainty in the current calibration environment at Clearfield. 

If the metrology Laser beam is not aligned with the axis of travel of 
the calibration fixture slider, the measured travel will be too small by 
an amount proportional to the cosine of the angular misalignment. The 
resultant error is called cosine error. By careful alignment, cosine 
error can be made as small as desired. A worst-case estimate for the 
PMD calibration fixture, corresponding to 1/16 inch beam offset in 144 
inches of travel results in 0.1 ppm cosine error 

We now turn to calibration-related mechanical errors, beginning with the 
PMD rotation axis. The radii! arm is mounted to a central shaft guided 
by a pair of annular contact bearings. The two bearings are pre-loaded 
and separated by approximately 4 inches Any radial motion, or 

"runout," of this bearing system will cause a one-for-one error in 

radius calibration. We have estimated this radial error motion to be 20 
microinches PV, which is the range of errors observed in precision 
machine spindles using ball bearings. The worst case diameter error is 
then 40 microinches. 
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If the radial errors of the spindle bearings are not in phase, then 
there will be a tilt error motion inversely proportional to the bearing 
separation. Assuming a 20 microinch radia) motion for each bearing and 
a 4 inch separation yields a tilt motion of 1° microradians or about 2 
arc-seconds. The actual radial error at the measuring tip depends upon 
the axial orfset of the tip from the center of rotation of the bearing 
system. Taking 6 inches as a typical offset yields a radial error due 
to tilt motion of 6 inches x 10 micro j.nches/inch = 60 microincher , or a 
diameter error of 120 microinches. 

We estimate a diametral error of 40 microinches due to radial motion and 
shape error of the measuring t i 4 contact bearing. Length calibration 
data for the inspection block, according to MSFC personnel, show a 
length uncertainty of -0/+200 mlcro'nches. We assume a least-count 
error in reading the digital linear gauge at each end of the calibration 
process, yielding a diameter error of 200 microinches. 

In this analysis we have assumed that the horizontal and vertical ball 
slides at the end of the PMD radial arm are in the same locations at 
each end of travel in the calibration process. Should this not be the 
case, errors contributed by these slides would have to be included in 
the calibration error budget. These errors are discussed in the section 
on mechanical errors. 

We have also assumed negligible error due to lack of squareness between 
the spindle rotation axis and the line of motion of the calibration 
fixture slider. The fixture is carefully adjusted during calibration so 
that the tip contacts the inspection block at the same height at both 
ends of travel. if these heights were to differ by 0.05 inches, the 
resultant diameter error would be about 10 micr^inches. 

Finally, we assume here that the tip contact forces during calibration 
are negligibly different from those encountered during part measurement, 
so that tip deflections are constant and are included in the values of 
the calibrated radii. 

The PMD measuring tips are bolted into place, with no attempt to make 
their locations kinematically determinate. Furthermore , we are not 
aware of any torque specifications in the tip mounting procedures. We 
estimate that each tip can be removed and re -mounted with a radial 
uncertainty of 100 microinches, leading to a diameter error of 200 
microinches . 

The calibration error budget is shown in Table 1. Combining the 
individual errors according to Equation (4) yields AD cal = ±702 
microinches . 
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TABLE 1. 


CALIBRATION ERROR BUDGET 


Error Source 


Peak- to -Valley 
Diameter Error 
(microinches) 


1. User Interferometer 


2 . 


a. 

Vacuum Wavelength (0.1 ppm) 

14.4 

b. 

Pressure Error (1 ppm) 

144.0 

c . 

Temperature Error (1 ppm) 

144.0 

d. 

Humidity Error (0,1 ppm) 

14.4 

e . 

Air Composition (1 ppm) 

144.0 

f. 

Cosine Error (0.1 ppm) 

14.4 

Mechanical Factors 


a. 

Spindle radial motion 

40.0 

b. 

Spindle tilt motion 

120.0 

c . 

Tip bearing errors 

40.0 

d. 

Tip mounting repeatability 

200.0 

e . 

Inspection block calibration 

200.0 

f. 

Linear gauge least count 

200.0 


^total - 1275 /jin. 


RMStotal <Eq. 3) = 129 /jin. 


AD ca l = ±702 _/jin. 
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The mechanical error budget includes those error components associated 
with the geometry and moving elements of the PM D. Spindle errors, tip 

bearing errors, and tip mounting repeatability have been discussed in 
the context of calibration errors and they also contribute to errors 
during measurement. 

Radial deviations are measured using a digital linear gauge with 
100 microinch resolution. Specifications provided by MSFC do not state 
the accuracy of this gauge, but experience with similar devices at the 
National Institute of Standards and Technology (NIST) suggests an 
absolute accuracy of approximately 200 microinches per inch of travel. 
With this estimate of gauge accuracy the diameter error is 400 
microinches, assuming a total travel of one inch during part 
measurement . 

The linear gauge/bail slide/measuring tip assembly of the PMD is 
characterized by what is called an Abb6 offset. This means that the 
linear gauge is not in line with the displacement to be measured, i.e., 
the radial motion of the measuring tip. Because of this offset, any 
angular motion (pitch) of the horizontal ball slide as it translates 
will cause a radial error. The magnitude of the error is simply the 
offset distance times the slide pitch error motion. The offset distance 
depends upon which tip is being used, with a maximum value of about 
8 inches. The pitch motion of the horizontal slide is not specified in 
the data sheet supplied by MSFC but may be estimated to be 5 arc-seconds 
in one inch of travel, a value typical of precision ball slides. Using 
these values, we estimate the radial error contribution to be 
8 inches x 5 arc-seconds x 5 microinches/inch/arc -second or 200 
microinches, with a corresponding diameter error of 400 microinches. 

Straightness error motions of the vertical ball slide cause one-for-one 
errors in radial data. The data sheets for this s’ ide specify a 
straight line accuracy of 500 microinches per Inch of travel. Assuming 
1/2 inch of travel during a part measurement implies a possible radial 
error of 250 microinches or 500 microinches diameter error. 

If the vertical ball slide is not mounted orthogonally to the horizontal 
slide, then motion of the vertical slide will contain a component in the 
radial direction. Estimating a squareness error of 20 arc-seconds and 
1/2 inch of travel yields a radial error of 1/2 inch x 20 arc-seconds x 
5 microinches/inch/arc - second = 50 raicroinches radial error, 100 
microinches on diameter. 

PMD radial error is also sensitive to angu**r motion of the vertical 
ball slide. T .I the slide rotates, or pitches, the measuring tip will 
move radially by an amount equal to the tip length times the pitch 
angle For a pitch of 2.5 arc-seconds in 1/2 inch of travel, and a 6 
inch tip, the error is 6 inches x 2.5 arc-seconds x 5 
microinches/inch/arc-second = 45 microinches radial error, 90 

microinches on diameter. 
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This estimated mechanical error budget is shown in Table 2. Combining 
the individual errors according to Equation (4) yield* = ±1061 
micro inches . 


5. THERMAL ERROR 

The coefficient of thermal expansion of an engineering material is 
defined by 


dL/L 

a(T) = , (7) 

dT 


where dL/L is the fractional change in a characteristic linear dimension 
and dT is the change in tempera^ ire . If a sample has length L^ at 
temperature T 0 , then the length L at the temperature T is found by 
integrating Equation (7): 


L = I, exp 


r J"a(T)dT 

I T* 

A o 



( 8 ) 


In common engineering practice a(T) is approximated by its average value 
a over the temperature range T-T 0 , so that Equation (8) becomes 


L = Loexp 



(9) 


The range of temperatures encountered in dimensional metrology is nearly 
always such that 5(T-T 0 )«1, so that Equation (9) may be written 


L = Lo 


1 + q (T~T 0 ) 




( 10 ) 


Equation (10) is the standard expression used to correct dimensional 
measurements for the effects of thermal expansion. [Note: we have 

replaced o by a for simplicity.] 
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TABLE 2. 


MECHANICAL ERROR BUDGET 

Peak- to-Valley 

Error Source Diameter Error 

(microinches) 

1. Spindle radial motion 

2. Spindle tilt motion 

3. Linear gauge accuracy 
U. Horizontal slide pitch 

5. Vertical slide straightness 

6. Vertical slide pitch 

7. H-V slide squareness 

8. Tip bearing errors 

9. Tip mounting repeatabi lity 

^total = 1890 nin . 
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For measurements that demand a very high degree of accuracy, it is 
important to realize that the arguments of Equation (10) may not be 
known exactly. Taking the differential of both sides yields 

6L = ciLq 6 (T-T 0 ) + Lod-To^a 


6 


= qLq I 6 (T-T 0 ) + (Wa)(M 0 ) 

n 




= ol 


Lq 6T-6T 0 + {Sa/a) (T-T 0 ) | 


( 11 ) 


Here, S T and S T c are temperature uncertainties and Sa is the uncertainty 
in the coefficient of thermal expansion. Since the signs of the 
uncertainties are generally unknown, the maximum error in estimated 
length is found by re-writing Equation (11) using absolute values: 

” a ^i [ \ST\ + |*T 0 | + {Sa/a) |T-T 0 Q . (12) 


The quantity <Sa/a is the fractional uncertainty in the nominal 
coefficient of thermal expansion. It is rarely encountered in 
dimensional metrology but it becomes significantly important for large 
parts (large qLq ) and for large temperature extrapolations (large 
|T-T 0 |). The consequences of this uncertainty are emphasized in Section 
20.2 of ANSI Standard B89.6.2 (Appendix A). 

Thermal expansion corrections for PMD measurement data employ two 
coefficients: (a) for the radial arm, a ARM = 13 ppm/°F (6061-T6 

aluminum alloy) and (b) for the part, a pAR ~ = 6.8 ppm/°F (steel;. The 
uncertainties in these coefficients are not known; they are estimated 
here to be 5% of the nominal values. That is 

^ q arm/ q arm = ^ q part/ q part ~ 0-05. ^13) 

Eased upon ANSI B89.6.2 these estimates are reasonable, and, in the case 
of the radial arm, may be somewhat optimistic. A brief survey of 
engineering material reference data on 6061 -T6 wrought aluminum alloy 
found values of the expansion coefficient ranging from 12 ppm/°F ^ALCOA 
Structural Handbook] to 13.5 ppm/°F [Standard Handbook for Mechanical 
Engineers]. Perhaps the definitive reference work is Volume 12 of the 
Therraophysical Properties of Matter, "Thermal Expansion of Metallic 
Elements and Alloys," which gives a value of a for 6061-T6 allov of 
12,5 ppm/°F at 20°C (68°F), with an explicit statement of a 

uncertainty in thermal expansion data on this alloy. 

Case diameter data from the PMD involves two thermal expansion 
ealeu 1 ations . Radial data is corrected for expansion (or contraction) 
of the radial arm between the calibration temperature (T AC ) and the arm 
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temperature during measurement (T AM ). Then, the part dimensional data 
is extrapolated from the part temperature during measurement (T PM ) to 
mtT, anmiov t" Vi a standard expression, Enuation (10), 

oo r. incsc — r--j • * 

using nominal values for the expansion coefficients. The uncertainties 
in the calculations can now be estimated using Equation (12) and 
Equation (13). We write for the two uncertainties: 

ad arm = diameter error due to arm thermal expansion uncertainty 

ADpart = diameter error due to part thermal expansion uncertainty. 

Using Equations (12) and (13) gives 


T 


AD 


ARM 


°ARM C R0M 



I + I*t am | 


+ 0.05 



(14a) 


and 

ad part 


°part d hom 



+ 


0-05 |T pm 



(14b) 


Here, D H0M is a nominal part feature diameter, taken to be 144 inches. 
Using the nominal values of o ARM (13 ppm/'F) and o PART (6.8 ppm/ F) , we 
have 


AD aRM = 1872 



I + I 6T am 1 


+ 0.05 



(15a) 


AD part = 979 [7sT pm | + 0.05 |T PM -6s7] . < 15 

In these expressions the temperature units are’F and the diameter errors 
are given in microinches. 

Once the various thermal environments are known (or at least estimated) , 
the total system error due to thermal effects can be calculated from 
Equations (15a-b), The two expressions are PV estimates of separate 
thermal error components. The total thermal error AD THERM is then 
calculated using Equations (4) and (5). 

Because of the very wide range of thermal environments found at 
Clearfield, there is clearly no unique value for the measurement thermal 
error We therefore proceed to calculate the thermal error for a range 
of temperatures typical of. the current environment. We will also 
analyze a "best-case" situation assuming that the entire PMD measurement 
process takes place in a thermal environment similar to that found in a 
standards laboratory. 

We first consider three thermal environments at nominal average 
temperatures of 68, 78, and 88’F. The 20'F range of temperatures is 
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representative of the range actually experienced at Clearfield during 
PHD measurement testing. The PHD radial ana and the measured parts have 
temperature sensors used for data correction during calibration and 
measurement. While these sensors have been calibrated at MSFC, in order 
to account for calibration errors, drift, linearity errors, and unknown 
temperature gradients , we *ssume an uncertainty 6T of 1*F (PV) for the 
average arm and part temperatures. That is: 

«T ac = «T am = «Tp M = ±0.5-F 

We now proceed to calculate the thermal error estimates. Recall that 
the PMD is calibrated at a nominal temperature of T AC and that the part 
is measured with the arm at temperature Tam and the part at temperature 
T PN . The quantities T AC , T AM , and T PM can assume any of the three 
chosen temperature values, although we expect that the PMD and the part 
are in thermal equilibrium during a measurement, so that Tam “ T pm . 
There are three special cases t n consider: 

( a ) T ac = T* m . If the temperature is nominally 

unchanged through a calibration/measurement cycle, 
then no thermal expansion correction is applied to 
the PMD radial arm. Because of the ±0.5°F 
temperature tolerance, however, the two temperatures: 
may differ by as much as 1°F. In this case is 

calculated using Equation (10) with T-T 0 = 1°F, 

q = a ARM , and Lq = D N0M = 144 inches. 

(b) T pm = 68°F. If the part is measured at a nominal 

temperature of 68 °F, then no part temperature 
correction is applied. Because of the ±Q.5°F 

temperature tolerance, however, the actual part 
temperature may differ from 68°F by as much as 
0.5°F. In this case AD paRT is calculated using 
Equation (10) with T-T 0 = 0.5°F, a = a PART , and 

Lq = D nom = 144 inches. 

(c) In all other situations the thermal errors are 

calculated using Equations (15a-b). 

Table 3 displays, in matrix form, the range of estimated total thermal 
error. We see that this error can be less than 0.0015 inches when gauge 
calibration and part measurement are carried out at 68°F with reasonable 
control (±0.5°F). On the other hand, the error is more than 0 003 
inches if the gauge is calibrated at 68°r and the measure nent is done at 
88 “F. 

As a "best-case" scenario, if the calibration and measurement process 
could be performed in a thermal environment of 68±0.2*F (typical of a 
dimensional metrology laboratory) , then the system thermal error would 
be less than 0.0007 inches. 
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TARir 3 


THERMAL ERROR MATRIX 


Measurement 

Temperature 

Tam * t pm 

CF) 


Calibration Temperature, T AC (°F) 



68±0 . 5 

78±0.5 

88±0 . 5 



68+0.5 

±1461 

±2061 

±2663 

78±0 . 5 

±2323 

j 

±1730 

±2323 

88±0 . 5 

±3188 j 

±2597 

±2104 


Table entries are estimated thermal error AD TH£RM in 
units of mioroinches. The method of calculation is 
explained in the text. 


NOTE: If T aC = T am = T PM = 68±0.2°F, then AD THERM = 693 microinches. 
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6 . SOFTWAR E ERRPB 


The PMD analysis software uses a simple chord algorithm to compute the 
perimeters of measured SRM features. The input data set of polar 

coordinate data pairs is first fit to a least-squares circle and the 
points re-expressed in a polar coordinate system with its origin at the 
center of this circle. This is mathematically equivalent to removing 
the first harmonic term in a Fourier series and has the effect of 

removing the PMD centering error. 

It should be noted that the data centering fit is performed 

independently for each measured feature, so that the PMD does not 

measure the complete joint geometry in a fixed coordinate system. If 
two SRM joint features were perfectly circular but eccentric, the 
eccentricity would not be detected. Because of the way in which the SRM 
cases are manufactured this is probably not a problem, but the effect 
should be recognized. 

Part diameters computed using the MSFC algorithm have been shown to be 
insensitive to PMD gauge centering. In one witnessed test, the gauge 
was deliberately de-centered by 0.25 inches and the measured feature 
diameter repeated to 100 microinches 

We have performed extensive testing of the MSFC chord algorithm, using 
computer -generated simulated case data as well as actual hardware 
measurement data supplied by MSFC personnel. Computed diameters using 
the chord algorithm were also compared with those determined by a 
variety of other curve fitting algorithms, including least-squares 
circles, linear arcs, blended smoothing polynomials, cubic B~splines, 
and global fitting to Fourier series up to the 16th harmonic in 6 . In 
addition to these tests we also examined the effects of resolver angular 
errors and radial noise (random radial errors) on the fit results. We 
summarize these tests below. 

We first tested the chord algorithm using computer-generated data 
representing an ellipse wi^h a semi-major axis of 74 inches and a semi- 
minor axis of 70 inches. The perimeter of this simulated case can be 
calculated to any required precision using high-accuracy numerical 
integration. The test data set had 3600 points, so that the numerical 
correct j.on factor of 1.000012693 was changed to 1.00000012693. In this 
test, Mie chord algorithm yielded the perimeter with an error of less 
than ! microinch. The actual numbers are (a) exact result (to 9 
significant figures): 452.476613 inches; (b) MSFC chord algorithm: 

452.4766124 inches. For comparison purposes we note that the perimeter 
of the best-fit circle is 452*563841 inches, which is too large by 
0.087 inches. The diameter of this circle is approximately 0.014 inches 
larger than that of the circle obtained by "rounding" the elliptical 
case . 

We then compared the chord algorithm with a range of other techniques 
for perimeter estimation, using actual SRM case measurement ^ata (MSFC 
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data file 07248709.49). This was a clevis feature using PMD tip #5. In 
our opinion, the most accurate of these algorithms is a global Fourier 
series fit to the data with terms included up to the 16th harmonic. The 
results of this comparison are as fellows: 


Algorithm 

16th-harmonic Fourier 
MSFC chord summation 
B- spline smoothing 
Linear arcs 
Laast-squares circle 


Perimeter (inches) 

454.242822 

454.242851 

454.242839 

454.242795 

454.240667 


Diameter (inches) 

144.589981 

144.589990 

144.589986 

144.589972 

144.589295 


The difference in mean diameter between the MSFC algorithm and the 
Fourier expansion fit is seen to be only 9 microinches. We also note 
that the least-squares circle result, while reasonably good in absolute 
terms, is the least accurate of the algorithms. This has been found to 
be generally true, particularly as actual case features depart 
substantially from circular, because of the averaging nature of the 
circular fit. 


The next test was designed to examine the effects of random radial 
errors on the calculated feature diameters. The size and distribution 
of random errors in real PMD data is not known, but an examination of 
the raw data from repeated runs suggests a level of 200-300 microinches. 
For this numerical experiment the PMD data from the previous comparison 
was artificially corrupted by the addition of a uniform distribution of 
random "noise" to the radial data. We then computed the mean feature 
diameter using the chord algorithm. The results, for various levels of 
noise, are as follows (average of 10 trials for eac*^ noise level): 


Radial Noj,?e 

( inches) 

0 

± 0.001 
± 0.002 
±0.003 
±0 . 004 


Mean Diameter 
( inches) 

144.589990 

144.590039 

144.590070 

144.590251 

144.590515 


Delta Diameter 
(inches) 

0 

+0.000049 

+0.000080 

+0.000261 

+0.000525 


It is clear from these results that the MSFC chord algorithm is 
insensitive to any plausible level of random radial error in the PMD 
data. 


As a final test of the analysis software, we examined the effects of 
angular errors in the data. The existence of such errors was suggested 
by calibration data on the resolver of PMD S/N 002. This data showed a 
second-harmonic angle error with an amplitude of approximately ±0.25 
degrees. We d'' not know the accuracy of this data, which was taken 
using radial case pin holes as angle reference locations. Clearly, the 
effects of angular errors will depend on the actual shape of the 


O 


17 



measured case features. A perfectly circular case, for example, would 
look circular regardless of the angular locations of the measured radii. 

We modelled a set of angle errors approximating the calibration data and 
used this computer-generated error set to corrupt the part measurement 
data from MSFC d^tJ file 07248709.49. The angular error function was 
the second harmonic form 


4 - ^max sin(20), (16) 

where 4(6) is the angle error at a nominal angle $ and ^ MAX is the 
maximum error. At each nominal angle the error 4 was added to 6 to 
yield the angle used in the fitting algorithm. The results of this 
procedure are as follows: 


. v (deg) 

Mean Diameter (inches) 

Delta Diameter (inches) 

0 

144.5900 

0 

+0.1 

144.5905 

+0.0005 

+0.2 

144.5910 

+0.0010 

+0.3 

144.5916 

+0.0016 

+0.4 

144.5921 

+0.0021 

+0.5 

144.5926 

+0.0026 


The results for negative values of <^ MAX show diameter changes of equal 
magnitude and opposite sign to those above. We see here that resolver 
errors can affect the measurement results at the level of 0. r 01 inches 
if the measured errors are accurate. This suggests that the PMD 
resolver systems be more accurately calibrated or be replaced by more 
accurate angle transducers. 

Based upon these studies and numerical experiments, we estimate that the 
maximum diameter error due to the MSFC analysis software is 
AD S0F t = ±1500 microinches. We note that the contribution of resolver 
error is not really a software error but we have included it here since 
the effects of such error are not readily apparent until the PMD data 
has been reduced and analyzed. 


7 . ES TIMATED TOTAL SYSTEM ERR OR 

The PMD measurement accuracy can now be estimated by combining the 
separate diameter error estimates according to equation (6). As 
explained in Section 5, the wide range of thermal environments 
encountered during calibration and measurement precludes any unique 
value for the system accuracy. Accordingly we present three estimates: 
a worst case, a best case (given the current state of temperature 
control) , and a potential accuracy achieved by better temperature 
control . 


9 % 
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A . WORST CAS E .ESTIMATE 

PMD calibrated at 68±0.5°F, part measured at 
88±0 . 5°F. 

From Equation (6) : 


AD- 


to 


TAL - ± Q 


(702) 2 + ( 106 1 ) 2 + (3188) 2 + (1500) 


D' 


= ±3746 microinches 


B. BEST CASE ESTIMATE 

PMD calibrated and part measured at 68±0.5°F. 
From Equation (6) : 


ADtotal - ± Q 


(702)2 + (1061)2 + (1461)2 + (1500) 




= ±2450 microinches 


C. POTENTIAL ACCURACY 


PMD calibrated and part measured at 68±0.2°F. 
From Equation (6) : 


AD 


TOTAL 


= ± ( 


( 6 3 5 ) 2 + ( 1061 ) 2 + ( 693) 2 + (1500) ; 


= ±2064 microinches 


8. DISCUSSION 


The error budget formalism used in this analysis can serve as a valuable 
tool for potential PMD accuracy enhancement. Many of the individual 
error components could be measured and the error estimates re-calculated 
based on the measurement results. One clear example of this procedure 
is the angle measuring resolver system. This system could be accurately 
calibrated and an error map created. This map would then provide a 
look-up table for data correction. Alternatively, the resolver could be 
replaced by a high-accuracy absolute optical encoder so that angular 
error would be negligible. The important point is that a means exists 
for a quantitative analysis of PMD error reduction as more information 
becomes available. 
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FORE WORT 


American National Standards Committee B89 on Dimensional Metrology, organized under the 
procedures of the American National Standards Institute, was formed to develop certain minimum stand- 
ards for the various parameters in metrology and represents the consensus of United States industry. The 
various subcommittees of Committee B89 deal with the different parameters, i.e., environment, angle, 
length, geometry, etc. Subcommittee B89.6 is assigned the task of developing standards in physical environ- 
ment and the effects of this environment and other extraneous influences on accuracy and precision of 
dimensional measurements. This standard for temperature and humidity is the work of the ANSI B89.6.2 
Working Group. Tlie results of its cooperative efforts are expressed in this document. 

The effect of heat flow and resulting temperature gradients, differences and variation from measure 
ment to measurement can result in errors of dimensional measurement because of the thermal expansion 
properties of materials. By international agreement the true size and shape of an object is that which 
exists at a uniform temperature of 68 ° F (20° C). The purpose of this standard is to provide American industry 
with pracfival requirements, procedures, and methods by which the intent of the international agreement 
can be satisfied without compromise to economical operation. 

In discharging its responsibilities, the Working Group has recognized two basic needs of industry. 
First, it recognizes the need for standard approaches to the buying and selling of artificially controlled en- 
vironments. Second., it recognizes the need for the qualification of individual measurements regarding errors 
induced by non-ideal temperature conditions 

Standard specifications for artificiaiiy controlled environments, in terms of the 4 >ality of temperature 
control, are especially necessary as a means of communicating metrological requirements to construction 
agencies such as heating and air-conditioning contractors. In specific instances, sufficient experience has 
been obtained such that required dimensional accuracies can be translated directly intc temperature control 
specifications. However, the Working Group has concluded that no general set of temperature control 
specifications can be stated th,t will simultaneously assure levels of measurement accuracy and avoid the 
risk of overdesign or underdesign. Indeed, no recommendation can be made on which type of artificial en- 
vironment or even whether one is necessary or not, that would represent the most satisfactory engineering 
for every application Consequently, the Working Group has chosen to list those properties of an artificially 
controlled environment that must be specified for an adequate description, to specify standard procedures 
for the administration of the required specifications, and to provide advisory information in the form of 
guidelines that the users of this standard may find helpful in the development of specifications adapted to 
individual needs. 

The metrologist, his management, or a potential customer of a metrological service has, each for his 
own purpose, a need and a right to know the magnitude of measurement errors induced by the thermal en- 
vironment. Therefore, this standard includes a description of procedures for the estimation of the error con- 
tributions caused by various defects of the thermal environment. Further, there is a need for a convenient 
means of communication between these parties. For this purpose, the Working Group has provided a stand- 
ard figure of merit, the Thermal Error Index. Because this document, for the first time, presents the Thermal 
Error Index for use by industry at large, the methods for its determination and use are carefully developed 
in an appendix. 

Recommendations for the control of humidity in metrological environments are included in this 
document, because it is often directly affected by and related to the control of temperature, especially in 
the design of room enclosures. 

After approval by the B89 National Standards Committee and submittal to public review the Stand- 
ard was approved by ANSI as a National Standard on October 30, 1973. 


iii 


9 



AMERICAN NATIONAL STANOAROS COMMITTEE B89 
DIMENSIONAL METROLOGY 


(Th» following I* the rootor of the CaaMsittee o( iht tint of approval of this ffdarg) 

OFFICERS 


E. C. Laewen, Chairman 

J. K. Emory, I*f Vico-Choirman J. C. Moody, 2nd Vico~Choirman 
Mory Hoskins, Executive Secretory 


AEROSPACE INDUSTRIES ASSOCIATION OF AMERICA, INC. 

K. J. Laight, Metrology Section, Primary Standards Laboratories, Hughes Aircraft Ccmpany, Culver City, California 

AMERICAN ORDNANCE ASSOCIATION 

J. C. Moody, Saadis Corporation, Albuquerque, New Mexico 

e « 

AMERICAN SOCIETY FOR QUALITY CONTROL 

John Novotny, Sperry Gyroscope, Greet Neck, New York 

AMERICAN SOCIETY FOR TESTING AND MATERIALS 

H. J . Stromba, Associate Director, Technical Operations, AST&M, Philadelphia, Penns y Iranis 

AMERICAN SOCIETY OF MECHANICAL ENGINEERS, THE 

I. H. Fullmor, Mt* Dors, Florida 

F. J . Meyer, Jr,, Machine Tool Engineering Associates Intemstional, Foreetdale, Rhode Island 
P. A . Smith, Professor, Massachusetts Institute of Technology, Cambridge, Massachusetts 

INSTITUTE OF ELECTRICAL fc ELECTRONIC ENGINEERS 
0. 5. Lonohon, Bloomfield, New Jersey 

INSTRUMENT SOCIETY OF AMERICA 

J. M. Comero#i, Notional Bureau of Standards, Washington, D.C. 

Herbert France, Stanley Works, New Ikitain, Connecticut 

L. N. Combs, Altomoto , E.L DuPont de Nemours M Company, Incorporated, Wilmington, Delaware 

NATIONAL MACHINE TOOL BUILDERS ASSOCIATION 

ft. 0. C/egg, Kearney M Tracker Corporation, Milwaukee, Via con a in 

SOCIETY OF MANUFACTURING ENGINEERS 
J. A. Coriolt o, Essex Junction, Vermont 

V. £. 0ieM, Shelton Metrology Laboratory, Paducah, Kentucky 

U.S. DEPARTMENT OF THE A® FORCE 

ft. L. Martin, A a roe pace Guidance & Metrology Center, Newark, Ohio 

U.S. DEPARTMENT OF THE ARMY 

M. L» Frve<.h*,’*'chf, Army Metrology fr Calibration Center, Redstone Arsenal, Alabama 
0. M. Solovoi, Edge wood Arsenal, Maryland 

T. W. Kane, Quality Assurance Directorate, Watervliet Arsenal, Watervliet, New York 

ft. 0. Smock, Physical Standards Laboratory, Army Metrology & Calibration Center, Redstone Arsenal, Alabama 
J, M. Me Kinloy, U.S. Army Development Ai Proof Services, Aberdeen Proving Ground, Maryland 

U.S. DEPARTMENT OF COMMERCE -NATIONAL BUREAU OF STANDARDS 

A. G. Strong, Optical Physics Division, National Bureau of Standards, Washington, D.C. 

0. 0. Sponger) berg. At tomato. Novel Weapons Engineering Support Activity, Washington Navy Yard, Washington, D.C. 
U.S. DEPARTMENT OF THE NAVY 

£. ft. Johnson, Chief of Naval Material, Department of the Navy, Washington, D.C. 

J, N, Comotto, Naval Ship Systems Command, Washington, D.C. 

UNIVERSITY OF CALIFORNIA 

J. 0. Bryan, Lawrence Livermore Laboratory, Livermore, California 


9 ' 


V 



•pvpmai vt trT*ir rrwDOBATtnM 

5« Pa CWtwt, Ptdtnl Electric Corporation, Kennedy Space Center, Florida 

ERGS CORPORA TIOW 

Dale W. Fmyberg, Engis Corporation, Horton Qrov«, Illinois 
ft J. Roilly, A /remote, Eafia Corporation, Morton Grovn* Dlinoia 

GURLEY TELEDYNE 

Ralph Goisor, Research & Development Laboratory, Gurley Teledyne, Troy, 

New York 

ME TROLOWICS STANDARDS LABORATORIES 

J, A. Hnrrifipten, Metro Ionics Standards Laboratories, Burbank, Calif crma 

CUMMINS ENGINE COMPANY 

M. E. Haskins, Cummins Engine Company, Columbus, Indiana 

AUTOWETICS— NORTH AMERICAN AVIATION, INCORPORATED 
J . A. Halt, Autoaetics, Anaheim, California 

MOORE SPECIAL TOOL COMPANY, INCORPORATED 

A, E. John* on, Moore Special Tool Company, Incorporated, Bridgeport, Connecticut 

A. W. Yeung, A /remote, Moore Special Tool Company, Incorporated, » id g* port, Connecticut 

FEDERAL PRODUCTS CORPORATION 

T. L. John ton, Jr ., Research fc Development, Federal Products Corporation, 

Providence, Rhode Island 

C. Whitney, A /remote. New Products Division, Federal products Corporation, 

Providence, Rhode Island 

TIC VAN KEUREN COMPANY 

R. W. Lamport, Tbs Van Kauren Company, Watertown, Massachusetts 
BAUSCH fc LOMB, INCORPORATED 

E. G. Loewon, Gratings fc Metrology Research, Bauach & Lomb, Incorporated, Rochester, New York 
GENERAL ELECTRIC COMPANY 

W. 0. McCollum, Knolls Atomic Power Laboratory, Schenectady, New York 
SPERRY GYROSCOPE 

John Novotny, Sperry Gyroscope Company, Great Neck, New York 

GREENFIELD TAP & DIE -UNI TED -GREEN FIELD DIVE ION OF TRW. INCORPORATED 
H. W. Porker, Greenfield Tap fr Die, Greenfield, Massachusetts 

INTERNATIONAL BUSINESS MACHINE CORPORATION 
A Ivin Millar, I.B.M., Endicott, New York 

STANDRIDGE GRANITE CORPORATION 

Roy' Stondndga, Standridge Granite Corporation, Whittier, California 

THE BENDIX CORPORATION 

Leo Ttchochflin, Tool A Gage Iri pact ion & Control, The Bend lx Corporation, Kansas City, Missouri 

F. W. Witxfce, Automation h Measurement Division, The Bendix Corporation, Dayton, Ohio 

BROWN fc SHARPE MANUFACTURING COMPANY 

E* L. Wcrte/et, Shown fr Sharpe Manufacturing Company, North Kingston, Rhode Island 

L. S. STARRETT COMPANY 

G. 0. Webber, Webber Gage Division, L~S. Starrett Company, Cleveland, Ohio 

INDIVIDUAL MEMBERS: 

E. M. Paome, Nuevo, California 
J. H. Wortben, Warwick, Rhode Island 
J . A. Emery, Weston, MaseachueetU 

R. P. Trowbridge, General Motors Technical Center, Warren, Michigan 

R. L. Eskert, Automation k Measurement Control, The Bendix Corporation, Dayton, Ohio 

E. E. Liodbarg, Hewlett Packard Laboratories, Palo Alto, California 

J, C. Comhi, Engineering Development Laboratory, E« I* DuPont de Nemours & Company, Wilmington, Delaware 


0 \ 
-d 


VI 


PERSONNEL OP SUBCOMMITTEE f UN ENVIRONMENT 


J . C. Moo4y, Cboirmon, Sand is Laboratories, Albuquerque, .w Mexico 
Merry Hosfcins, Secretory, Cummins Engine Co., Columbus, Indiana 

J. 0. Aryan, University of California. Lawrence Livermore Laboratory, Livermore, California 
J. C. Combi, Enfpg. Dev. Laboratory, E. I. DuPont de Nemours fk Co., Wilmington, Delaware 
A. M. Dexter, Jr., United Nuclear Corp., Uncesville, Connecticut 

M. L. Five cb ten #cbt. Army Metrology & CaJ'^ation Center, Redstone Arsenal, Alabama 

J. A. Harrington , National Astro Laboratories, s. Burbank, California 

M J. Leighf, Primary Standard* T Soratory, Hughes Aircraft Co., Culver City, California 

£. R. McCluro, University ol California, Law-er»ce Livermore Laboratory, Livermore, California 

J. M. McKinioy , Aberdeen Proving Grounds, Aberdeen, Maryland 

J. W. Nob/e, Mason As Hanger * Silas Mas*. a., Inc., Amarillo, Texas 

J. J. Schoonover, Internet! o M Business Machines, Endicott, New York 

A. D. Skufca, Newark Air Force Static", Newark, Ohio 

L. C. WhitNtn, Jr., Union Carbide Nuclear Co., Oak E Tennessee 

J. L. Wilson, Sandia Laboratories, Livermo r* _ulif .1 

A. W. Young, Moore Special Tool Co., Inc., Bridgeport, Connecticut 



AMERICAN NATIONAL STANDARD 
TEMPERATURE AND HUMIDITY ENVIRONMENT 
fOR DIMENSIONAL MEASUREMENT 


ANSI 889.6.2-1973 


AMERICAN NATIONAL STANDARD 
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1. SCOPE AND INTENT 

This standard is intended to fill industry's need for 
standardized methods of : 

a. Describing and testing temperature-controlled 
environments for dimensional measurements, and 

b. * Assuring itself that temperature control is 
adequate for the calibration of measuring equipment, 
as well as the manufacture and acceptance of work- 
pieces. 

Z REFERENCED DOCUMENTS 

2.1 Standards and Specifications 

This standard has been coordinated insofar as pos- 
sible with the following standards and specifications. 
Unless stated otherwise, the latest issue is implied. 

2.1.1 Governmental 


Don, and Air-Conditioning Engineers, 345 East 47th 
Street, New York, New York 10017 


3. DEFINITIONS 

3.1 Average Coefficient of Expansion 

The average coefficient of expansion of a body 
over the range of temperature from 68° F (20° C) to t 
is defined as the ratio of the fractional change of 
length of the body to the change in temperature. 

Fractional change of length is based on the length 
of the body at 68° F (20° C). 


a (68, t) - 


Ju ~ i 

( t - 68 ) 


(l) 


Hereinafter the term “coefficient of expansion" 
shall refer only to the average value over a range from 
68 F (20 C) to another temperature, t. 


a. M!LC-45662A-Calibration System Require 
ments 

b. MIL-HDBK-52-Evaiuation of Contractor’s Cali 
bration System 

c. MIL-Q-9858A-Quality Program Requirements 

d. Fed. Std. #209-Clean Room and Work Station 
Requirements, Controlled Environment. 

2.1.2 Non -governmental 

a. Standards of the American National Standards 
Institute (ANSI), formerly United States of America 
Standards Institute (USASI), 

b. Standards of the American Society for Testing 
and Materials (ASTM), 

c. Standards of the Society of Automotive Engi- 
neers, Inc. (SAE), 

d. Recommendation RI -Standard Reference Tem- 
perature for Industrial Length Measurements, Inter- 
national Organization for Standardization (ISO). 

2.2 Other Publications 

a. ASH RAE -Handbook of Fundamentals pub- 
lished by the American Society of Heating, Refngera- 


3.2 Coefficient of Expansion 

The true coefficient of expansion, a, at a tem- 
perature, /, of a body is the rate of change of length 
of the body with respect to temperature at the given 
temperature divided by the length at the given tem- 
perature. 



3.3 Comparator 

Any device used to perform the comparison of the 
part and the master is called a comparator. 

3.4 Differential Expansion 

Differential expansion is defined as the difference 
between the expansion of the part and the expansion 
of the master from 68° F (20° C) to their time-mean 
temperatures at the time of the measurement. 

3.5 Differential Response 

Differential response is defined as the relative 
length vanation between any two objects per unit 
sinusoidal environment temperature oscillation as a 
function of frequency of temperature oscillation 
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3.6 FoR ScMt Dtltfomt try 

Full-scale dilatomctry is a procedure for deter- 
mining the average true coefficient of expansion of a 
workpiece. 

3.7 Dfcncmioful Response 

Dimensional response is defined as the amplitude 
of absolute length variation of ca object per unit of 
sinusoids 1 environment temperature oscillation as a 
function of the frequency of temperature oscillation. 

3i Drift Test 

An experiment conducted to determine the actual 
drift inherent in a measurement system under normal 
operating conditions is called a drift test. Since the 
usual method of monitoring the environment (see 
Definition 3.13) involves the correlation of one or 
more temperature recordings with drift, the test will 
usually consist of simultaneous recordings of drift 
and enviromnentd temperatures. The recommended 
procedure for the conduct of a drift test is given in 
20.3.1. 

3.9 Master 

The standard against which the desired dimension 
of the part is compared is called the master. The 
standard may be in the form of the wavelength of 
light, the length of a gage block, line standard, lead 
screw, etc. 

3.10 Mastering 

The action of nulling or setting a comparator with 
a master is called mastering. 

3.11 Mastering Cy^e Time 

The time between successive mastering* of the 
process is called the mastering cycle time of the 
process. 

3.12 MtMuremtnt Cydt Time 

The time between measuring and the previous 
mastering is called measurement cycle time. 

3.13 Monitoring 

To ensure the constancy of the Thermal Error 
Index (see 3.22), it will be necessary to monitor the 
process in such a wiy that significant changes in 
operating conditions are recognizable. 

The recommended procedure is to establish a 
particular temperature recording station or stations 
wh’ch have a demonstrable correlation with the mag- 
nitude of the drift. 

The temperature of the selected station should be 
recorded continuously during any measurement 
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process In wIim-Ii flic !«• he ;;pp!;c;! !! mil 

recording shows a significant change ol conditions, die 
index is null and void for that process, and a rc- 
eviluation of the index should be conducted, or the 
conditions corrected to those for which the index 
applies. 

In addition to continuous monitoring of environ- 
mental conditions, it is recommended that efforts be 
made to establish that the process is properly soaked 
out. This may be done by checking the temperature 
of ail elements before and after the execution of the 
measurements. 

3.14 Nominal Coefficient of Expansion 

The estimate of the coefficient of expansion of a 
body shall be called the nominal coefficient of expan- 
sion. To distinguish this value from the average co- 
efficient of expansion a (68 f r) it shall be denoted by 
the symbol x . 

3.15 Nominal Differentia Expansion* 

The difference between the Nominal E ipansion of 
the part and of the master is called the Nominal Dif- 
ferential Expansion: 

NDE = (NE) part - (NEW. (3) 

3.16 Nominal Expansion* 

The estimate of the expansion of an object from 
68° F to its time-mean temperature shall be called the 
Nominal Expansion, and it shall be determined from 
the following relationship 

NE*x (£)(/- 68). (4) 

3.17 Part or Workpiece 

In every dimensional or geometric measurement 
process, there is usually some physical object for 
which a dimension is to be determined. This object is 
called the part or workpiece. 

3.18 Soak Out 

One of the characteristics of an object is that it has 
a thermal “memory". When a change in environment 
is experienced, such as occurs when an object is 
transported from one room to another, there will be 
some period of time before the object complete^ 
“forgets” about its previous environment and ex- 
hibits a response dependent only on its current en- 
vironment. The time elapsed following a change in en- 
vironment until the object is influenced only b> the 
new environment is called soak out time. After soak 


•These concepts are us*d in determining the Thermal Lrror 
Index m Section 6 
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out, uic object u niu io be in equilibrium with t he 
new environment. In ctaes where an environment is 
time variant, the response of the object is also a 
variable in time. 

119 Te mper a ture otf a Body 

3.19.1 Temperature at a Point. When discussing a 
body which does not have a single uniform tempera- 
ture, it is necessary to refer in some manner to the dis- 
tribution of temperature throughout the body. Tem- 
perature at a point in a body is assumed to be the tem- 
perature of a very small volume of the body 
centered at that point. The material of which the 
body is composed is assumed to form a continuum 

3.19.2 The Temperature of a Body. When the dif- 
ferences between the temperatures at all points in a 
body are negligible, the body is said to be at a uniform 
temperature This temperature is then the tempera- 
ture of the body. 

3.19.3 Instantaneous Average Temperature of a 
Body. When the body is not at a uniform temperature 
at all points, but it is desirable to identify the thermal 
state of the body by a single temperature, the tem- 
perature which represents the total heat stored in the 
body may be used. When the body is homogeneous, 
this is called the average temperature of the body 
(This temperature is the average, over the volume of 
the body, of all point temperatures.). 

3.19.4 Time-Mean Temperature of a Body. The 
average of the average temperature of a body, over a 
fixed period of time, is called the time-mean tempera 
ture of the body. The fixed period is selected as ap- 
propriate to the measurement problem 

3.20 Temperature Variation Error, TVE 

An estimate of the maximum possible measure- 
ment error induced solely by deviation of the environ- 
ment from average cone tions is called the Tempera- 
ture Variation Error. TVE is determined from the 
results of two drift tests, one of the master and com- 
parator and the other of the part and the comparator. 

3.21 Thermal Conductivity 

Thermal conductivity is normally defined as the 
time rate of heat flow through unit area and unit 
thickness of a homogeneous material under steady 
conditions when a unit temperature gradient is main- 
tained in the direction perpendicular to area. In this 
standard it is designated by K and has the units of 
BTU/hrft 2 °F 
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3.22 i narmai Error index 

The aunmation, without regard to sign, of the es- 
timates of all thermally induced measurement errors, 
expressed as a percentage of the working tolerance 
(or total permissible error). 

3.23 Expansion 

The difference between the length (or volume) of a 
body at one temperature and its length (or volume) at 
another temperature is called the linear (or volumetric) 
thermal expansion of the body. 

3.24 Thermally Induced Drift 

Drift is defined as the differential movement of 
the part or the master and the comparator caused by 
the time variations in the thermal environment. 

3.25 Tima Constant of a Body 

The time required for a physical quantity to change 
its initial (zero-time) magnitude by the factor (1 - Me) 
when the physical quantity is varying as a function of 
time, /(f) according to either the decreasing ex- 
ponential function, 

or the increasing exponential function, 

f(0 * 1 

when k - 1/r, it is called the time constant of the phy- 
sical quantity. In this standard it is designated by r. 

Since e has the numeric value 2.7 1 828—, the change 
in magnitude (1-1/e) has the fractional value 
0.6323 2—. Thus, after a time lapse of one time con- 
stant, starting at zero-time, the magnitude of the phy- 
sical quantity will have changed approximately 63 2 
percent. 

The time constant of a body can be used as a 
measure of the response of the body to environ- 
mental temperature changes. It is the time required 
for a body to achieve approximately 63.2 percent of 
its total change after a sudden change to a new level 
ir its envuonment. 

3.26 Transducer Drift Check 

An experiment conducted to determine the drift in 
a displacement transducer and its associated amplifiers 
and recorders when it is subjected to a thermal en- 
vironment similar to that being evaluated by the drift 
test itself. The transducer drift is the sum of the 
“pure” amplifier dnft and the effect of the environ- 
ment on the transducer, amplifier, and so on. The 
transducer dnft check is performed by blocking the 
transducer and observing the output over a period of 
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lime ill least ax long ax the duration of the drill text to 
he peril nmeu. iiioekir.g s transducer involves making a 
transducer effectively indicate on its own frame, base, 
or cartridge, in the case of n cartridge-type gage head, 
this is accomplished by mounting a small cap over the 
end of the cartridge so the plunger registen against the 
inside of the up. Finger-type gage heads can be 
blocked with similar devices. Care must be exercised 
to tee that the blocking is done in such a manner that 
the influence of temperature on the blocking device is 
negligible. 

3.27 Uncertainty of Nominal Coefficient 
of Expansion 

The maximum possible percentage difference be- 
tween the true coefficient of expansion, a, and the 
nominal coefficient of expansion shall be denoted by 
the symbol 5 . 

5 . 100— % (5) 

a 

This value, like that of k itself, must be an esti- 
mate. Various methods can be used to make this esti- 
mate. For example, 

(a) The estimate may be based on the dispersion 
found among results of actual experiment s con- 
ducted on a number of like objects; 

(b) The estimate may be based on the dispersion 
found among published data. 

Of the two possibilities given above, (a) is the 
recommended procedure. 

Because the effects of inaccuracy of the estimate 
of the uncertainty are of second order, it h con- 
sidered sufficient that good judgment be used. 

3.28 Uncertainty of Nominal Differential Expansion 

The sum of Uncertainties of Nominal Expansion 
of the part and master is called the Uncertainty of 
Nominal Differential Expansion. 

UNDE = (UNE) |Wt ♦ PEU (6) 

3.29 Uncertainty of Nominal Expansion 

The maximum difference between the true thermal 
expansion and the nominal expansion is called the Un- 
certainty of Nominal Expansion, it is determined 
from 

UNE = *i('-68) D 


•See Equation 23, Paragraph 20.2 for pouible revision 
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4. GENERAL REQUIREMENTS 

# 

A 1 The methods of describing and testing icm pc ra- 
ture<on trolled environments shaft be in accordance 
with Section 5. 

4.2 A calibration, part manufacture, or part ac- 
ceptance procedure complies with this standard if u 
is carried out with all pertinent components of the 
measurement system at 68° F; or if it can be shown 
that the Thermal Error Index (as defined in Section 
6) is a reasonable and acceptable percentage of the 
working tolerance. 

5. DESCRIPTION AND TESTING OF 

ENVIRONMENT 

In this section an environment is to be understood 
as a room, box or other enclosure through which a 
temperature-controlled fluid (liquid or gaseous) is 
circulated and which is intended to contain dimen- 
sional measurement apparatus. 

5.1 Description of Environment 

In the following paragraphs the essential properties 
of an environment are listed. These characteristics 
must be unequivocally specified. 

5.1.1 Thermal Specifications. The following prop- 
erties of a controlled environment must be specified. 

5. 1.1. 1 Cooling Medium. The type of coohno 
medium is to be described in terms of its chemical 
composition and physical properties of viscosity, den- 
sity, specific heat and thermal conductivity When 
common substances such as ambient air or water are 
to be used, unless otherwise specified, their properties 
are to be assumed those given m standard tables 

Commercial fluids such as oils may be specified b> 
manufacturer and type. 

5. 7. 7.2 Flow Rate and Velocity. The flow rate of 
the cooling medium shall be specified in units of 
weight per unit time, volume per unit time, or 
changes per unit time. Veioc^y shall be spccucd m 
feet per unit Umc. 

$.1.1.3 Ranges of Frequencies of Temperature 
Venation and Limit from Mean Temperature. These 
two properties are interrelated and cannot be specified 
separately. For example, in general, the higher the 
frequency the wider the permissible temperatu-- ex- 
cursions from the mean temperature in the cooling 
medium (see Section 10). Frequencies are to be 
specified in cycles per unit time, and limits from mean 
temperature in plus or minus (±) units Fahrenheit 
(units Celsius). Separate limit specifications may be 
applied to a number of frequency ranges. 
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5. 1. 1.4 Mean Temperature. The mean temperature 
thill be the time average temperature at a specified 
poini (or time average of the average of temperatures 
as more than one specified point) within the bounda- 
ries of the environment. A period of time over which 
the time average if to be computed shall be specified. 

5. 1. 1.5 Gradients. Within the wo*. olume of 
the environment, maximum steady-state temperature 
differences are to be specified. The specification can 
take one or both of two forms: 

(a) “Worst case” maximum temperature difference 
in the cooling medium between any two points within 
the specified boundaries of the environment; 

(b) Maximum rate of change of temperature along 
one or more specified directions within the specified 
boundaries. 

Specifications can be applied to sub-boundaries, or 
volumes within volumes. 

Each such specification should he qualified as to 
the conditions of acceptance testing, e.g., whether or 
not equipment and/or personnel are to be within the 
boundaries during the testing. 

5.1.2 Humidity. Requirements for humidity con- 
trol arise from desires to provide human comfort, to 
prevent deleterious effects of moisture such as corro- 
sion of workpieces and measurement apparatus, and to 
maintain measurement accuracy of workpieces that 
are dimensionally sensitive to moisture (for example, 
certain hygroscopic materials). Specifications for 
humidity control shall l>e consistent with the practices 
established by the American Society of Heating, 
Refrigeration and Air-Conditionmg Engineers 
(ASHRAE). 

5.1.3 Maintainability. Requirements shall be speci- 
fied governing the maintenance of perf >rmance in ac- 
cordance with the above requirements in order that 
deterioration of the environment with time, due to 
the reduction of contioi efficiency, can be held within 
acceptable limits by implementation of established 
operating and maintenance procedures. 

5.2 Testing of Environments 

5.2.1 Thermal Specify rations 

5.2. 1. 1 Cooling Medium. When a cooling medium 
other than air or water is to be supplied as a- part of 
the environment, its thermal properties must be 
qualified. The standard test methods listed here may 
be used to determine the required properties. The list 
is not intended to be exhaustive, but is only 
representative of the many standard procedures 
available . 
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5.21.1.1 Viscosity 

ASTM D445-65 - Viscosity of Transparent and 
Opaque Liquids 

ASTM D! 545-63 - Viscosity of Transparent Liq- 
uids by Bubble Time Method 

5.2. 1.1. 2 Density (Specific Gravity ) 

ASTM D94 1-55 (Reapproved 1968)-Density and 
Specific Gravity of Liquids by 
Lipkin pycnometer 

ASTM D 1 298-67 - Density, Specific Gravity or 
API Gravity of Crude Petro- 
leum and Liquid Petroleum 
Products by Hydrometer Meth- 
od 

5.2. 1.1. 3 Thermal Conductivity 

ASTM D2717 68T— Thermal Conductivity of Liq- 
uids 

5.2. 1.2 Flow Rate and Velocity 

5. 2. 1.2.1 Row Rate 

ASTM D2458-69 - Flow Measurement of Water 
by the Venturi Meter Tube 

ISOR541-1967 - Measurement of Fluid Flow by 
Means of Orifice Plates and 
Nozzles 

5.2. 1.2.2 Velocity 

ASHRLAE Handbook lists several accepted test 
methods. 

5.2. 1.3 Ranges of Frequencies of Tempers :ure 
Variation and Limits from Mean Temperature Limits 
of variation f/om mean temperature in the cooling 
medium at any specified point or points within the 
specified boundaries is to be determined by use of a 
sensitive recording thermometer. The time constant 
of this instrument is to be no more than one-fifth 
( 1 /5) of the period of the shortest cycle period (highest 
frequency) of interest; and its resolution is to be at 
least one-tenth (1/10) of the smallest amplitude of 
specified temperature variation. 

The temperature recording duration shall be a mini- 
mum of 24 hours and should be as long as the 
representative work cycle (e.g., a week). The test 
should be performed under worst-case conditions (i.e.. 
hottest day of year and coldest day of year). 

The maximum peak -to-v? Hey temperature varia- 
tion is to be determined from the recorded data for 
every discernible frequency component. Isolated dis- 
turbances, i.e., single “spikes”, are to be regarded as a 
component of the appropriate frequency. 
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Limits fr<*n mean for each frequ en; r y CiniipiillVJli 
art* lo he calculated as one-half 06) of Use observed 
peak-to-vallcy excursion. 

5.2. 1.4 Mean Temperatures. A thermometer that 
has been calibrated by comparison with a standard 
pUtinu ( tv resistance thermometer m the speckled cool- 
ing medium is to b* used to determine mean tempera- 
ture. A recording thermometer whose output is 
averaged over the test period is preferred. However, a 
thermometer with a large time constant can be used if 
it is read 2t a frequency corresponding to one-half 
(4) of its time constant over the test period and all 
such readings averaged. 

The resolution of the test thermometer shall be 
one-tenth (1/10) of the specified tolerance for the 
mean temperature. Also, the test thermometer should 
be standardized or calibrated so that in use it would 
indicate temperatures with an inaccuracy no worse 
than one-fourth (V4) of the specified tolerance for the 
mean temperature. 

5.2.1. 5 Gradients. If a “worst case” specification 
is to be administered, the locations of the temperature 
sensors are to be dearly specified. 

If a maximum rate of change of temperas - per 
unit length in * given direction or directions is »o be 
administered, a grid pattern shall be established to 
define the locations of temperature sensors. In the 
case of room, box, and tank enclosures, that fraction 
of the total volume immediately adjacent to the en- 
dosure walls is to be excludtd from the gnd pattern. 

dess otherwise specified, for an enclosure that is 
empty, does not contain furniture, personnel, and 
equipment other than that used in the performance of 
the test, the excepted volumes shall be those obtained 
by reducing each dimension by 10 percent. For 
example, a 10' x 10' x 10' room shall be reduced to 
9' x 9' x 9', the excluded volume being that contained 
within 6" from the walls, ceding, and floor 

if the specification calls for testing with equipment 
o personnel in the enclosure, the specification shall 
include a description of excluded areas or volumes ad- 
jacent to such personnel or objects. 

Testing shall be performed over a representative 
work period 

5.2.2 Humidity. Humidity in the controlled en- 
vironment is to be measured by any means having suf- 
ficient accuracy to satisfy the design specifications 

A sling psychromstcr provides an economical and 
convenient way to monitor humidity on a periodic 
basis 
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6. ASSURANCE of adequacy of environ 

MENTAL CONTROL-THERMAL ERROR 

INDEX 

In the buying and selling ol goods and services, it 
is common practice that ihc buyer requests the seller 
to produce evidence of his ability to satisfy specifica- 
tions. Most questions of met rological capability can be 
satisfied by means of available certification services, 
e.g., gage block calibration by a competent agency 
in the special case of thermal effects errors, it is some 
times possible for the seller to meet the buyer’s re- 
quirements by showing that work is to be done in an 
environment of a description acceptable to the buyer. 
In general, however, the variety of gages, workpieces, 
and measurement procedures that is found in practice 
precludes having all requirements met by even a small 
choice of standard environment descriptions. 

The purpose of this section of the standard is to set 
forth a procedure for the assessment of the maximum 
possible measurement error due to all ther-.ai effects 
Basically, this procedure consists of computing esti- 
mates of possible error, taking into account un- 
certainties involved in the computations, and experi- 
mentally determining other eiror components. Sum- 
ming all component error estimations gives an esti- 
mate of the over-all maximum possible error. Dividing 
this number by the total jxTinissiblc error from all 
sources (working tolerance) gives an index of men! 
(Thermal Error Index) that can be used for the ad 
ministration of environment quality control require- 
ments. 

In file following paragraphs, iJrle standard proce- 
dure for computing the Thermal Error Index is de- 
scribed. OrJy the most significant components of error 
are considered. Approximate methods of error esti- 
mation are used. Section 20 of the Appendix includes 
a more thorough discussion of the methods of error 
estimation and possible means of improving the ac- 
curacy of the estimates. However, the following are 
the standard procedures. 

6.1 Consaquencos of Environment Mean Temperature 
Other then 68° F (20° C> 

6.1.1 Length Measurements 

6.1 1.1 Nominal Expansion, NE. Assuming that an 
object in a oonvective environment has a uniform 
temperature equal to the environment mean tempera 
ture in its immediate vicinity, its Nominal Expansion 
is estimated by 

NE = x (L)(t - 68). ( 8 ) 

6 1 1.2 Nominal Differential Expansion, NDE The 
difference between the Nominal Expansion of die part 
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6.1.2 M«*urtment$ Other than Length. In cases 
of measurements other than length, where a standard 
computational method if not given, the estimation 
of the consequences of environment mean tempera- 
tures other than 68° F (20° C) shall be made according 
to formulae appropriate to the specific case, if pos- 
sible. An example is given in Section 20 of the 
Appendix. 

Satisfactory formulae with which to carry oa; the 
intent of this section are not always available because 
of insufficient technical knowledge of each specific 
case. 

6.2 Consequences of Uncertainties of Error 
Computations 

6.2.1 Length Measurement 

6.2. 1. 1 Uncertainty of Nomine! Expansion , UNE. 
The Uncertainty of Nominal Expansion is 

UN£=k (5 )(L)(t- 68)/ 100%* (10) 

where r is the worst-case (greatest difference from 68) 
measured temperature of the object (part or master). 

"Rearrangement of E/?iurion 7, Paragraph 3. *9. 

The fact that it is mandatory l .lake use of direct- 
ly measured temper?. cures in the calculation of Un- 
certainty of Nominal Expansion means that the intent 
of Section 6.4 is automatically satisfied fir the case 
of length measurements. 

6.2. 7.2 Uncertainty of Nomina / Differentia! Ex- 
pansion. UNDE. The sum of the Uncertainties of 
Nominal Expansion of the part and master is called 
the Uncertainty of Nominal Differential Expansion. 

6.2.2 Measurements Other than Length. In cases 
of measurements other than length, where a standard 
computational method is not given, an estimation of 
the possible consequences of the uncertainties of co- 
efficients of expansion and temperature measure- 
ments shall bt made according to formulae appro- 
priate to the specific case if possible. An example is 
given m Section 20 of the Appendix. 

Satisfactory formulae with which to carry out the 
indent of this section are not always available because 
of insufficient technical knowledge of each specific 
case 


6.4 Lons«qu«ncoi of Time Variation of Tamparature 
Langtft Measurements, Temperature Variation 
Error, TVE 

The maximum observed or recorded thermal drill 
during cither part/comparator or master/comparator 
drift test (see Appendix), whichever gives the larger 
value, during a period of time corresponding to the 
measurement cycle is called the Temperature Variation 
Enor. 


6.4 Consaquenctt of Gradients in Environment 
Temperature 

In some measurements, gradients in environment 
temperature have a resultant error effect that is 
distinct from those described in the preceding para- 
graphs and which is significant enough to be given 
special consideration. For example, in the use of sur- 
face plates the consequences of an environment mean 
temperature other than 68° F (20° C) are ins.gmficani 
provided the material of the surface plate is sufficient 
ly homogeneous with respect to thermal expansion 
properties. However, the control of temperature di f - 
feienccs between the lop and bottom of the surface 
plate is of prime importance. 

No general formulae can be given for the estimation 
'vf gradient temperature effects. 


6.5 Thermal Error Index, TEI 

The sum of all the approximate thermal effects 
error components from Paragraphs 6.1. 6 ?, 6.3. and 
6 4, where such paragraphs apply, expressed as a 
percentage of the total permissible error from all 
sources is called the Thermal Error Index 

For length measurements, 


NOE. UNDE. TVE „ , w (||J 
Total permissible error 


if no correction for differential ex pans on is attempted, 
and 


TE] UNDE + TVE 

Total permissible error 


I00^r 


( 12 ) 


if a correction for differential expansion is computed 
and applied in the measurement procedure 
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APPENDIX 


Historical Background of Standard Temperatures for Dimensional Standards 1 


The presently recognized standard temperature of 
20° Celsius (68° Fahrenheit) was preceded by at least 
two others, namely, 0° Celsius (32° Fahrenheit), the 
temperature of melting ice, and 16-2/3° Celsius (62° 
Fahrenheit) which historically, at least, appears to 
have been the earliest. 

Zero Celsius was the temperature adopted by the 
International Congress for Weights and Measures at 
Pans in 1889 at which the platinum-indium b^r. 
maintained at the International Bureau cf Weights and 
Measures at Sevres, France, represented the meter 
exactly However, its length at 20° Celsius had been 
carefully determined and for many years the com- 
parisons of the various national standard meter bars 
with the international standard bar were made at 20° 
Celsius. 

France, however, continued to use 0° Celsius as the 
standard temperature for length standards and gages 
until 1931, when 20° Celsius was adopted inter- 
nationally 

Sixteen two-thirds° Celsius (62° Fahrenheit) was 
long recognized as the standard temperature for di- 
mensional standards m Great Britain, dating back to 
at feast 1831 Like France, Great Britain switched to 
?0° Celsius in 1931 

As early as 1898-99, C. E Johansson of Sweden 
consiJcrcd that the measurements giver for his meas- 
uring instruments should apply at 19 or 20° Celsius 
Since he found the usual temperature was between 1 5 
and 25° Celsius, he took the mean value, 20° Celsius, 
as the temperature easiest to maintain and generally 
prevailing m daily workshop practice 

About 19U-1, Johansson had a nominally !00- 
millimeier rod measured by the International Bureau, 
which stated that the rod was 1 00 millimeters at 
20.63° Celsius With this information, Johansson made 


' This information was taken, in part from unpublished notes 
of Irvin H Pullmer, former Chief of the Engineering 
Metrology Section The National Bureau of Standard'. 

1 ’’The Calibration and Dimensional Changes of Precision 
Gage B'ocks ' C G Peters and H S Hn\J American 
Machinist September JO and October 7 , ( V20 


a new rod 0.0007 millimeters longer than the first so 
that it would be of correct length at 20 3 Celsius. This 
rod was then used m subdividing lengths to produce a 
set of gage blocks m various senes. 

From this beginning, the use of 20° Celsius as a 
standard temperature grew until in April 1931 the 
International Committee A Weights and Measures 
adopted a resolution that, in the future, the tempera- 
ture of 20° Celsius (68° Fahrenheit) should be univer- 
sally adopted as the normal temperature of adjustment 
for all industrial standards of length. Consequently 
most of the nations of the world adopted this tem- 
perature as their standard for length adjustments 
Thereafter, standards of length were adjusted to be 
nominally correct at 20" Celsius and manufacturers of 
gage blocks, end standards, scales, tapes, fixed dimen- 
sional gages, lead screws, etc adjusted their manu- 
facturing methods so these device: would be nominal- 
ly correct at 20° Celsius. 

In an article by Peters and Boyd of the National 
Bureau of Standards published in I920 2 wc find (he 
statement, ‘The temperature at which the auual 
length of the gage equals the nominal length must 
therefore, be specified and is usually taken as 20 ( or 
68° F " 

The National Bureau of Standards installed its first 
constant-iemperature (20° Celsius) room for call h r j t 
ing gage blocks and other dimensional standards m 
1924 Johansson, himself, was responsible for the first 
industrial room, rt the Ford Motor Company, m 1 926 

Twent) Celsius thereafter became generally 
used that Recommendation No. I of the International 
Organization for Standardization, issued in 1954 
promulgated ;ts «%e among the 40 participating 
countries 

Therefore, since at least as far back as 19J2 when 
it was recorded that Johansson was making his gage 
blocks for America on the basis of 1 inch equals 
25 4 millimeters and nt a temperature of 20° Celsius 
millions of items such as gage blocks, end standards, 
micrometer, dimensional gages and products on of 
machines having lead screws, have been manufactur'd 
to 0e nominally correct at 20° Celsius 
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The argument frequently advanced that measure- 
ments can be made at any temperature and corrected 

V\° r /C3® C\ k>< Mwtluina rr»rr»rlinn« K*wH fw% 

«v i.v v vv* » / v; ■J'pv “*D — 

various coefficients of thermal expansions is valid 
only if the coefficient; are known with sufficient ac- 
curacy. The importance of this was known at least 
forty yean ago when, in the paper by Peters and 
Boyd of the NBS, referenced above, this statement 
was made. 

44 Another property which must be recognized whtn 
considering true accurate length of gages is the 
thermal expansion of the material A 1-inch steel 
gage (block) increases in length about 0.000 013 
inches for every degree C (Celsius) nse in tempera- 
ture. The temperature at which the actual length of 
the gage equals the nominal length must therefore 
be specified and is usually taken as 20°C or 6X°F 
Ai 25°C the length of a gape which is one inch at 
20°C is about 1 .000 065 inches. If c gage be mcav 
ured at the higher temperature us length at 20"C 
may be computed if the expansion coefficient is 
known. If higher precision is desired, it is not good 
policy to use expansion coefficients given in tables 
because our measurements show that the expansion 
coefficients of steel may vary from 0.000 0105 to 
0.000 0135 depending on its hardness and com- 
position. 

This vanauon would permit an unknown steel 
gage that agrees exactly with a standard at 25°C to 
differ from it by more than 0.000 01 inch at 20°C. 
If the unknown piece that is being measured is 
brass or some othei material having an expansion 
coefficient that differs greatly from that of the 
standard the effect of temperature change is 
augmented F»-om these considerations it is evident 
thai to measure or use gages with an accuracy in 
the millionth placr the coefficient of expansion of 
the material must be accurately known, and also 
temperature controlled and measured to at least 
0 IT ” 

A 1-inch micrometer caliper is required to have a 
maximum error in indicate^ reading not exceeding 
0.000 1 inch and a 12-uich micrometer caliper not 
exceeding 0.000 3 inch. If the standard temperature 
were 23° C (73.4° F), the existing i-inch micrometers 
would be in error by about one-third of their toler- 
ance while the 12-inch micrometers would be in error 
by about one and one-third times the tolerance 


* Thr Development of Enfine^nng Mefioiogy/’ by F K 
Roll, institute of Production Engineers. 1, 1952 
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An indication of the impact on the users of the 
thousands of precision machine to is using lead 

trrewc nrecfntly in fTictenrf rtf tnitj^ Ky rnn. 

adt ring the case of the standard leading screw lathe in 
use at the National Physical Laboratory in Great 
Britain. The 60 inch traveree of its lead screw is cor- 
rect to within 0.000 lOinches at 68* F. At 73.4 C F an 
additional error of 0.000 21 inches would be in- 
troduced, for a total error of about 0.000 31 inches 

In 1952, Dr. F. H. Rolt, Superintendent of 
Metrology at the National Physical Laboratory' had 
this to say about Great Britain's change from 16-2/3° 
C to 20° in 1931. 

“Previous to 1931, the standard temperature for 
engineering measurements (the temperature at 
which standard length gages and other gages arc ad 
justed to size) was62 F(16-2/3°C)in Great Britain. 
0°C in France and 20 C in some other European 
C'unirifs and the United States. V confused 
slate of affairs was abolished m A; *u 1^31 when 
the International Committee of Weights and Meas- 
ures adopted a resolution that, in future, the terr 
perature of 20°C (68° F) should be universally 
adopted as the normal temperature of adjustment 
for all industrial standards of length. This change 
was supported by the British Standards Institution 
and put into effect by the National Physical Labo- 
ratory at the beginning of 1932. It amounted to a 
change of approximately four and a half ten 
thousandths in the length of a 12-inch gage ana 
pro-rata for other lengths. In ether wo'ds.a 12-inch 
gage which had been true to size at the old tem- 
perature of 62°F had to be actually shortened 
closely 0.000 45 inches to bring it true to size a: 
the new temperature of 20°C 

British industry weathered this change without 
much trouble 20 years ago, but with the genera! all- 
round improvement in accuracy in latter years to 
effect a change of that magnitude today would 
bring quite a number of difficulties in its train 

Fc' many years, “°C” has stood for “degrees 
centigrade,” the well established temperature scale 
devised by the Swedish Astronomer Anders Celsius, 
1701 to 1744 In keeping with the practice of 
hononng certain individuals who have made sig- 
nificant contributions to our scientific knowledge 
and development, by renaming units after them 
(e.g., “Hertz” for “cycles per second”), the use of 
the word “Celsius” over “centigrade” is now tc be 
preferred. It is a fortunate coincidence that both 
begin with the letter “C” 
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10. ADVISORY INFORMATION PERTAINING TO 
np^RtPTinN O r Environments 

10.1 Dw criptwi o? Emrkonmtit i 

10.1.1 Thermal Guktolincs 

fCt 1. 1. 1 Cooling Medium. For most conventional 
metrology work, the appropriate cooling mediurr. will 
be air and the encloiure will be a room, in a building, 
in which human being* are occupied in daily tasks. 
However, in some cases it is advisable; to consid ;r 
fluids other thin air for the cooling medium. 

Considerations that may have influence in choosing 
a fluid other than air for the cooling medium are 

(-) Greater heat removal capacity, 

(b) More accurate temperature sensing and control, 

(c) Avoidance of contamination of parts, espe- 
cially from oxidation. 
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Both (a) and (h) can rcxull Imhh using ,t liquid 
(iiai lias tisc grealci ciieciivcncss as a cootaiii l lie cl 
fcclivcncss of a coolant is mcasn'cti m terms u! a film 
cocfflcicnl (A; Blu/hr-lt 1 * I*). In a spccilk case <>i 
heat flow from a surface into a fluid, the film co- 
efficient is a rather complicated function of fluid 
properties, flow velocity, and geometry. To simplify 
estimating the relative effectiveness, air and water 
have been chosen in Figure 1 as representative of all 
gaseous and liquid coolants respectively. Rough 
boundaries can be established for the film coefficients 
attainable for each of these coolants for natural and 
forced convection, as is shown by the vertical lines in 
Figure 1 . 

Ine expected film coefficient in slowly moving 
room air is about 1.0. This can be increased by in- 
creasing the velocity of the air up to a limit ai about 
h = 10.0, which is approximately the lowest limit for 


AIR: WATER: 

I 2 NATURAL CONVECTION 1 I NATURAL CONVECTION 
FORCED CONVECTION f~"l FORCED CONVECTION 



FIG I Coolmm-effecii^n^ts chert for nr versus writer to natural and forced convection for material thickness varying 
from 0 1 to IOC m The solid lines separate regions of part and coolant dominance for iron and a typical pia»t'C 
A condition within the area of coolant dominance indicates that improvement m rhe control of the part temper d 
ture can be achieved by increasing the flow or changing the coolant 
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witei in natural convection. Water in forced convec- 
ri™ has 2 potential film coefficient of several hundred 
(lines lhal of room ajr. 

The tola! resislance lo heat flow from a solid wall 
into a fluid coolant includes the thermal resistance of 
the wall material: 


R 


1 1 
*A h 



(13) 


for a wall cooled on both sides and heated at midplane , 
and 


R 


1 1 
"a h 



(14) 


for a wall heated on one surface and cooled on the 
other. 

The point at which an increase in the film coef- 
ficient begins to produce diminishing returns, is where 
the internal and external resistances are equal. The 
diagonal lines on the chart show the boundaries of this 
effect for different wall materials and one-sided and 
two-sided cooling. The chart shows that for one-sided 
cooling of iron, the forced water convection is fully 
effective for wall thicknesses well over I inch, while, 
for plastics, forced water convection is less effective 
because of the dominance of internal resistance for 
much thinner sections. 

10. 1. 1,2 Flow Rate and Velocity. The flow rate of 
the cooling medium is of prime importance in the 
control of frequency of temperature variation, and 
temperature gradients. Frequency of temperature 
variation in an artificially controlled environment is 
related to lags and delays ir the feedback control sys- 
tem and, thus, to flow rate and the distance between 
temperature sensor and the heating and cooling sur- 
faces. Gradients are related to the flow rate, the 
specific heat of the cooling medium and the magnitude 
of heat loads, as well as the distribution of the heat 
sources in contact with the flow. 

In general, the higher the flow rate the higher the 
frequency of temperature variation and *he smaller 
'ne temperature gradients In addition, the lug ler the 
flow rate the higher the velocity of the cooling 
medium. This can have both beneficial and detrimental 
effects. 

The beneficial effect of higher velocity's higher 
film coefficients. With higher film coefficients, a 
smaller temperature difference is required to remove 
heat from the surface of an object. This means that 
objects, with either internal heat sources (e g., motors 
inside machine frames) or receiving heat by radiation 
(e.g., from electrical lights) will have temperatures 


more nearly equal to that of the cooling medium if 
the velocity is increased. 

The cictrimcntal effect is a tendency to discomfort 
human personnel. Where human operators are ex- 
pected to work in rapidly moving air, the permissible 
velocity is limited because the humar response to a 
given dry-bulb temperature depends on the air 
velocity and the relative humidity. At a dry-bulb tem- 
perature of 68° F, a relative humidity of 50 percent, 
and a velocity of 100 fpm, the temperature is felt as 
63° F, and as 58° F for 400 fpm (see Figures 2 and 3) 
The maximum permissible velocity for human com- 
fort in a 68° F, 50 percent R.H.,room is about 25 fpm. 

High precision i metrology laboratory air velocities 
range from 6 to 20 fpm. 

10. 1. 1.3 Ranges of Frequencies of Temperature 
Variation and Limits from Mean. The dimensional 
response of an object to ambient temperature van- 
tion depends on its length, coefficient of expansion, 
and time constant (see 3.2.5). The time constant of an 
object can be estimated from 

CV 

time constant = r = — (15) 

hA 


where 


V= volume, cu. ft. 

A = surface area, sq. ft. 
h = film coefficient, Btu/hr. ft. 2 °F 
C = themuJ capacitance, Btu/°F ft 3 . 

Values for C are approximate 
iron, Steel — 54 
Aluminum ^ 36 
Brass ^ 48. 

Example: 

A steel gage block 1 inch square in cross-section 
and 10 inches long, in natural convection (estimate 

h = 2 ) 


T 


5 4(10) 

2(42)02) 


0.53 hr. 


This time constant is the time the gage block would 
take to reach 63.2 percent of its total change (see 
3.2.5). For example, it would be the time required for 
the object to change temperature 0.632 degrees after 
a step change in environment temperature of one 
degree. 

For a I -degree step in temperature m the air 
around the gage block of the above example, the 
gage eventually changes length approximate^ 60 


1 1 


1 

i 
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FIG 2 Ch»M lor determining elfective temperature lor sedentary individuals, normally clothed, from measurements ofl dry-bulb temp* r * 
lure, wet-bulb temperature arid velocity ol air 



WET BULB TEMPERATURE IN DEGREES FAHRENHEIT 
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DRY BULB TEMPERATURE IN DEGREES FAHRENHEIT 


FIG 3 Still'tir comfort chart of the American Society of Heating, Refrigerating and Air -Conditioning Engineers 
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micro! nches. But it takes 0.53 hour (32 minutes) to 
ch«Agc 3S mscrcinches. This ^own«o of response, or 
therms) inertia, is important to the specification of 
environments, because it means that high frequency 
temperature variation is tolerable. The higher the 
frequency, the more tolerable it is. 

Experience shows that most machinery, instrument 
stands, etc., have a bird ted range of volume to surface 
area ratio. Quite good results are often achieved with 
a frequency of temperature variation (in air) of 1 5 to 
60 cycles per hour and an amplitude of 1 degree 
Fahrenheit. For the gage block in the above example, 
this temperature variation would cause a length 
variation of less than 1 microinch. 

Like those of high frequency temperature varia- 
tions, the effects of very low frequency temperature 
variations are not significant when the part and 
master have closely similar dimensional response. 

The most fortunate case possible is that in which 
the part, master, and comparator (see Section 20) all 
have the same dimensional response characteristics. 
Then no temperature variation effect is significant. In 
general, however, there exists an upper and lower 
limit of frequency between which is a frequency of 
maximum differential response. Unfortunately, it is 
not uncommon that gaging systems have their maxi- 
mum differential response at a frequency close to the 
natural 24-hour day/night cycle period. 

Consequently, it is usually advisable to specify the 
tolerance-to-temperature variation in terms of allow- 
able deviations from mean temperature which vary ac- 
cording to the frequenev. Closer limits should be 
applied to low frequency components, and wider 
limits may be permitted for high frequency com- 
ponents. 

10. 1. 1.4 Mean Selection of a mean 

environmental temperature affects the cost of refriger- 
ation and heating equipment, insulation, and flow dis- 
tribution. 

Operation at a temperature other than 68° F (20° 
C) entails consequences in the form of potential errors 
of measurement that must be carefully evaluated. 
Evaluation procedures are described in Section 20. 

The most common objection to operating a room 
enclosure at 68° F, other than the cost of the air con* 
ditioning system, is a possible discomfort to person- 
nel. As discussed in Section 10.1.1.2, a high air 
velocity can cause a sensation of much lower tem- 
peratures and result in complaints. In order to main- 
tain human comfort without a requirement for 
special clothing, the velocities to which personnel are 
subjected should be less than 20 fpm to avoid the sen- 


sation of drafts. Conventional registers, at which the 
velocities may be 800 fpm or more are not satisfac- 
tory. Large inlet and outflow areas are recommended. 
Full-flow ceilings are used aiccessfully to simul- 
taneously provide high flow rate and low velocity. 

In cases where the needs of measuring equipment 
(68° F)and human personnel (low velocity) cannot be 
satisfied simultaneously, it is recommended that the 
equipment environment and human environment be 
separated. Uae of special air-flow boxes, liquid baths, 
or localized high-velocity air showers have been used 
successfully for this purpose. 

10. 1 . 1.5 Gndmnts. Gradients are the most difficult 
of all non-ideal temperature conditions to assess for 
possible error effects. The existence of gradients, of 
course, implies that portions of the environment will 
not be at the same mean temperature s'* that the con- 
sequences of mean temperatures other than 68° F 
(20° C) will be different in different locations in a 
room. Movement of equipment or workpieces from 
one area to another will result in a change in the error 
pattern. 

Machinery is affected by gradients in a variety of 
ways. For example, a machine with a high vertical 
column (z-motion) where the z-motion is controlled 
by a lead screw will have a progressive error if there is 
a high vertical temperature gradient. In addition, if 
the vertical slide carries a long cantilever arm, the arm 
will undergo a transient change of length when raised 
or lowered. 

Surface plates art affected by vertical gradients m 
that a temperature difference between the top and 
bottom of the plate will cause the plate to bend. For 
solid surface plates, the amount of bending or out-cf- 
flatness, 6, is calculated by using the following 
formula: 



( 16 ) 


where 

L = length of the surface plate 

H = height'or thickness of the surface plate 

T u = upper surface temperature) 

> A/ ~ T u - T\ 

f x = lower surface temperature I 
>c * coefficient of lin„«»r thermal expansion 
R * radius of curvature of the plate. 

Machine bedwavs are similarly affected b\ both 
vertical and horizontal gradients which cause angular 
motion* (pitch, roll and yaw). 
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Gradients occur because of heat sources that exist 
within the boundaries of the environment. For this 
reason, it is difficult to administer meaningful re* 
qui foments by testing in the absence of equipment 
and personnel that shall exist under normal working 
conditions. The main sources of heat are the electrical 
lighting fixtures, electrical and electronic equipment, 
motors, and people. Room enclosures with only the 
electrical lighting Fixtures present and operating have 
been tested with an observed gradient of less thin 
0.1° F per foot in any direction. However, the same 
room with equipment installed normally has gradients 
of over 0.2° F per foot and high gradients of several 
degrees per foot near the surfaces of surface plates, 
electronic cabinets, etc. 

As mentioned in Section 10.1.1.2, increasing the 
flow rate will decrease the gradients. For example, sur- 
face plates are observed to have temperatures on the 
upper surfaces of 2° F or more above the local mean 
in a flow rate of 10 to 1 5 changes per hour and a 0.5° 
F or less in 100 changes per hour. 

10. 1 .2 Humidity. In certain measurement systems, 
a significant error can occur if an incorrect value for 
humidity is used in computing a dimension. For 
example, in the measurement of the length of gage 
blocks by interferometry, a 10 percent relative 
humidity uncertainty will introduce an error of 0.1 
microinch per inch of length. Therefore, in labora- 
tories wnere these kinds of measurements are to be 
made, it is desirable to control (and measure) the 
humidity within close limits to keep this uncertainty 
small. 

There are three basic requirements for humidity 
control: 

(a) To provide human comfort, 

(b) To prevent deleterious effects of moisture such 
as coirosion of woikpieces and measurement ap- 
paratus, and, 

(c) To maintain measurement accuracy of work- 
pieces that are dimensionally sensitive to moisture. 

A discuss- of the latter requirement is felt to be 
beyond the scop* of this stand^.d since it would deal 
with parameters ol materials that arc not directly cor- 
relative with the guidelines for environmental control 
of dimensional metrology laboratories. 

Requirements 1 ami 2 are related, but are almost 
diametrically opposed. Consequently, there have been 
many suggestions given as to what hmits, if any, 
should be placed on the range of relative humidities 
to be permitted in dimensional metrology enclosures. 

For maximum protection against corrosion of fer- 
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rous components of measuring instruments, humidity 
should be maintained at very low levels. How: :r, for 
the maximum comfort of personnel m a laboratory 
where the dry bulb temperature is main Mined at 68° 
F, the humidity should be kept at a high level. The 
relative humidity recommended is a reasonable com- 
promise between these two extremes. 

Probably, the most frequently heard value for the 
upper limit is 45 percent relative humidity. A lower 
limit is then selected that is high enough to afford a 
certain degree of personal comfort, but low enough to 
not economically compromise the 45 percent upper 
limit. 

What is frequently overlooked, however, is that 
relative humidity, in itself, is just one of the con- 
tributing factors to corrosion. Of almost equal im- 
portance are the constituents of the atmosphere, or 
cooling medium, in which the workpiece and meas- 
uring apparatus are placed. For example, the presence 
of certain hygroscopic salts in the air will either cause 
or accelerate the corrosion of iron exposed to the air 
even though the relative humidity is relatively low. A 
saturated solution of lithium chloride will stand in 
equilibrium with air having only a 12 percent 
relative humidity. Similarly, saturated solutions of 
either calcium chloride or magnesium chloride will 
stand in equilibrium with air having a relative humidity 
of 31 or 33 percent relatively. Yet iron exposed to air 
containing any of these salt solutions will corrode 
more rapidly than if they were not present. Again, 
iron exposed to air containing ordinary solutions of 
sea salts shows little corrosion at a relative humidity 
of 30 percent, but fairly rapid corrosion if the air has 
a relative humidity of 35 percent. Salts arc not the 
only corrosion-acceieraitng agent m air, so far as iron 
is concerned. Small traces of sulphur dioxide (a com- 
mon constituent of industrial and urban air) will ac- 
celerate corrosion in iron at ordinary- temperatures 
and humidities. 

Probably one of ihe greatest accelerators of corro- 
sion in dimensional metrology laboratories is the per 
spiration residue deposited dunng handling. The 
chloride ion in the residue is probably the main ac- 
celerating agent, although the fatty acids will also be 
a factor. 

Since iron, in sorr . form, is the most common 
material found in the workpieces and measuring 
equipment in a dimensional metrology' laboratory, the 
rest of this section will deal briefly with the role 
humidity plays in its corrosion. 

Probably the most exhaustive and definitive labo- 
ratory studies on the atmosphenc corrosion of metals 
were those reported by W. H. J. Vernon in England 
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ihortiy after World War I. Briefly stated, these studies 
tfwwed that the rusting of iron in air was not neces- 
sarily associated with the dew-point as had been sup- 

I . L..4 : — * 1 - i— *U. 
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attack was identified with a critical humidity very 
considerably below saturation in an ordinary room 
atmosphere of low relative humidity, the process of 
rutting is influenced entirely by the suspended par- 
ticulate impurities in the atmosphere. Consequently, 
screening or protection of the surface from such mat- 
ter could arrest, or even prevent rust. However, even 
in the case of screened or filtered air, a primary oxide 
film wiD form even with relatively low humidities. 
The scxaHed “critical humidity" values for iron are 
approximately 62 percent and 82 percent. These are 
the relative humidity values at which profound in- 
creases in corrosion occur. Vernon’s studies showed a 
gradual increase in corrosion with increases in relative 
humidity from 0 percent to 99 percent. 

Based on Vernon’s studies of the atmospheric cor- 
rosion of metals and the studies conducted by, or for, 
the U.S. Navy on the long-term preservation of mate- 
rials (Operation Mothball), it is generally accepted 
that there will be little or no destructive corrosion of 
metals if they are held at 30 percent relative humidity 
in a reasonably pure atmosphere, i.c. t free of harmful 
particulate matter such as salts and sulphur dioxide. 
By using protective oils or greases on the base sur- 
faces, iron can exist without further corrosion at a 
relative humidity of 45 percent, provided the surfaces 
to be protected are cleaned as thoroughly as possible 
to eliminate possible hygroscopic dirt particles before 
the protective coating is applied. 

To reiterate, the presence of moisture is essential 
for natural corrosion to take place at normal tempera- 
tures, presence of hygroscopic matter on the surfaces 
can accelerate the normal rate of corrosion, and the 
presence of certain materials, such as salts, chloride 
ions, and fatty acids can precipitate corrosion at 
relative humidity levels well below saturation. 

An upper limit of 45 percent relative humidity has 
been suggested for dimensional metrology laboratories 
provided, of course, that normally bare iron surfaces 
are clean and protected with some type of coating 

10.1.3 Maintainability. Operating and maintenance 
procedures must be promulgated that, if followed, will 
ensure maintaining the performance of the dimen- 
sional metrology enclosure within its specified design 
limns 

10.2 Tarring 

The purpose of this section is to give users of tlm 
standard some understanding of the practical prob- 


lems that exist in the administration of the require 
ments outlined in Section 5. Wherever possible, 
specific procedural suggestions are made. However, a 
thorough d isc usser of *"* ' possible instrument ct 

procedure that is available for the administration of 
each requirement is beyond the scope of this docu- 
ment. 

It is strongly recommended that every specification 
include a description of the test procedure or instru- 
ment that is intended for the administration of each 
requirement. Section 2 of this document contains 
some of the more common sources of information on 
practical procedures and instrumentation. 

10.2.1 Thermal Guidelines 

10.2. 1. 1 Cooling Medium. Because this standard 
has been deliberately kept is general as possible in 
order to permit the use of a variety of cooling media, 
there is little that can be discussed here beyond that 
already presented in Section 5. The user of this 
standard is cautioned that the properties listed :n Sec- 
tion 5 are only those that pertain to the thermal be- 
havior of the cooling medium. For specific cases, other 
properties such as color, opacity, odor, loxjut>. 
acidity, lubricity, etc. may be very important. 

10.2. 1.2 Flow Rate and Velocity. As mentioned in 
Section 1 0.1. 1, air is I lie most widely used cooling 
medium in dimensional metrology laboratories or 
enclosures, although the possible use of a fluid 
medium in some enclosures cannot be overlooked. 

When air is used as the cooling medium, there is a 
tendency to have it flow at relatively low rates to pro- 
vide as high an effective temperature as possible for 
the occupants of the laboratory consistent with the 
68° F design specification. This tendency, aside from 
the effect on the film coefficients discussed in Section 
10.1.1.1 and 10.1.1.2, creates problems when meas- 
uring the air velocity. At velocities of 0 to 100 fpm, 
the flow pattern is frequently very unstable. As a re- 
sult, the mass turbulence level may be of the same 
magnitude as the velocity. Consequently, it becomes 
imperative that any instrument used be properly 
calibrated, and the using personnel be aware of both 
the limitaticn* of the instruments, and their opera- 
tion. For example, several types of thermal ane 
mometers (so-called hot-wire types) can be used in 
this range; but the accuracy of the measurements 
the lower end could be questionable, even though the 
precision of the measurements is quite good Another 
example. non-dsre:tional instruments are usua]l> un 
able to distinguish between large-scale turbulence and 
the mass velocity of air Tables 1 and 2 list various 
types of instruments commercially available to meas 
urt either flow’ rate or velocity. 
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The Reynold’s nuir referred to in Table 2 is a 
dimensionless pa/am' u.i used to designate the ratio of 
the inertia forces to the viscous forces in a fluid 
motion that occurs at the transition from laminar to 
turbulent flow. 

Measurements made using the kinds of instruments 
shown in Tables 1 and 2 are frequently made either in 
the ducts conveying the air, or in close proximity to 
such ducts. It should be recognized, however, that this 
may not give a complete picture of the actual air 
changes in the room since the air leakage into the 
laboratory caused by wind or temperature differences 
are not indicated by these means. Consequently, if a 
more accurate determination of the toiai change time 
is required, it irughl be necessary to go to u measure- 
ment system employing a thermal conductivity com- 
parator and a tracci ?: s. In tin:* system, a known 
amount of a tracer (usually some percent of the 
tor* 1 mr volume^ is released into the room and 
allowed to thoroughly mix with the air. As the 
,ciease occurs, this mixture becomes diluted. The con- 
ductivity comparator is then used to measure the de- 
crease m concentration at regular time intervals. 

The infiltration can then be calculated from 

c=q ) e- |rr/v (n) 

where 

C - concentration after t minutes, percent 

Q * initial tracer gas concentration, percent 
k = infiltration rate, cubic feet per minute 
v - volume of room, cubic feet 
c* 2.718. 

This infiltration rale can then be uscJ to correct 
(he flow rale yielded by itsin^ the more conventional 
instruments 

10.2. 1.3 Ranges of Frequencies of Temperature 
Variation and Limits from Mean. The main factors to 
be considered in choosing an instrument or instru- 
ments with which to administer temperature variation 
requirements are the frequencies of interest and the 
cooling medium. The instrument chosen must have a 
sensing element with a time constant small enough 
that the highest frequency of interest is detected and 
ui^played without significant attenuation or distortion. 

One point frequently overlooked is that a sensing 
element may have a different time constant lor each 
medium it is m. 

Example 

Bare Thermistor 

Time constant in air-3 minutes 

Time constant in Liquid —3 seconds 
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10.2. 1.4 Mean Temperature. Mean temperature is 
not measured directly (see S. 1.1.4). However, the 
temperature sensor-recorder system used to measure 
the temperatures from which the mean temperature is 
calculated must have adequate sensitivity, precision, 
and accuracy and must be used properly so that the 
calculated mean temperature will fall within acceptable 
confidence limits. 

10.2. 1.5 Gradients. The effect of thermal gradients 
can best be measured by closely monitoring the tem- 
perature of the master, the part, ano «.nc comparator 
during the actual measurement and applying the neces- 
sary thermal differential corrections to the measure- 
ment results. 

The composition and flow rate o! the cooling 
medium should be monitored for continued con- 
I or ma nee to design specifications. 

10.2.2 Humidity. Humidity is to be measured by 
any method having sufficient sensitivity and accuracy 
to assure the basic design specifications are met. 

10.3 Operation and Maintenance 

10.3.1 Thermal Guidelines 

10.3.1.1 Once the heat transfer into an enclosure 
has been established, it should hold fairly constant as 
long as the physical integrity of the enclosure is not 
disturbed. Some long-term shifts due to aging of the 
materials, such as the wall insulation, may be ex- 
pected. Normally, however, this should not pose a 
serious threat. 

Installation of beat-producing sources adjacent *o 
the controlled enclosure should be avoided il at ail 
possible because o! the possible effect on the heal 
transfer If a recalculation of the heal tra.islcr should 
show a significant change that could alfccl the thermal 
stability in the enclosure. additional insulation may 
be required. 

If the integrity of the enclosure is maintained and 
the condition of the filters, the lighting system, and 
the air -conditioning system is maintained at a suf- 
ficiently high level to minimize deviations in tempera- 
ture, cooling medium flow and velocity, little else 
should be required. 

10.3. 1.2 Flow Rare and Velocity. Periodic main- 
tenance of the cooiing medium distribution system is 
normally sufficient to maintain the established cooling 
medium flow rates and velocities provided the layout 
of equipment in the enclosure has not been sufficient- 
ly rearranged, or new instruments have been added 
that could disrupt the initial cooling medium flow pat 
terns. 


i * 
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10.3.1.3 Rangas of Frequencies of Temperature 
Umramai^ea* ArtW ^ n/iJ#T /mm kMmmn The ‘tnjy Wl e y 

lc»r the accuracy and p r ccision of iiie temperature 
sensing system to remain within the suggested limit* 
rs lor a regularly scheduled standardization and/m 
calibration program to be established and followed. 

A periodic mainten* e program is recommended 
for the temperature control system for the enclosure 
to assure that the destgL criteria are satisfied. 

10.3.1.4 M 99 n Twnpenture. If the procedures 
given in the other sections of 10.3.1 are followed, the 
established .,itan temperature should be maintained. 

10.3.1.5 Gradients. Because of the difficulties en- 
countered in establidting and maintaining thermal 
stability in a dimensional standards laboratory, a 
program of continued vigilance to ferret out causes 
of instability is strongly recommended. This is par- 
ticularly important during the periods when measure- 
ments are actually being made. 

10.3.2 Humidity. The specified humidity limits 
should be maintained by any suitable means. 


20. ADVISORY INFORMATION PERTAINING TO 
THE ASSURANCE OF ADEQUACY OF ENVI- 
RONMENTAL CONTROL 

!n this section it is assumed that the measuring 
equipment and the thermal environment exist, and 
that normal or expected operating conditions are in 
force. The object of the discussion is to describe the 
manner m which one goes about determining the ex- 
tent of measurement errors resulting from non-idea] 
temperature conditions. 

The ideas and methods described are those found 
in fairly common usage by metrologists everywhere. 
But, for the first time, these ideas and methods are 
unified and formally presented Some of the concepts 
presented nuy at first appear strange and unrelated to 
previous experience. The 3-element system concept, 
for example, will probably fall in this category. How- 
evei, with a little patient study, the concept will be 
seen to correspond to common notions, and us 
utility in a disciplined investigation will become clear. 

The other notion that may appear to be new »s that 
of the uncertainty o r the coefficient of expansion. 

Each of these concepts is examined and reduced to 
a practical procedure in the first four of the following 
paragraphs 

The last paragraph of this section is devoted to ex- 
plaining the Thermal hrr^r Index and us use 


ANSI WS.6.2-1973 


20.1 Estimation of Consequents of Mean Environ 
■Mfitd T amnirafii rat Other then 68 F (20 Cl 

20.1.1 Length Measurements. 'I he assessment of 
I lie consequences of icrnpcralurcs other th'^n I* 
(20 C) arc easily obtained by means ol equations tli.it 
give the Nominal Differential Expansion in terms o! 
the Nominal Expansions of (he part and master 


NDE = (NE) P - (NE) m 

(18) 

and 


NE = kL(T- 7i). 

(19) 

Combining these equations, we get 


NDE = k p L (T p - T,)-K m L(T m -T s ) 

(20) 

= L [xp [Tp — T s ) — K m [T m — T s )] . 


Assuming that the part and master both are at the 
mean temperature, T p * T m = T me (the only reason- 
able assumption unless thermometers arc attached to 
both the part and master), we sec that the error is 
reduced to insignificance if the coefficients of thermal 
expansion approach equality. And this is true even 
with a large deviation ol the mean environmental tem- 
perature from 68° F (20° f) 

liccausc the great majority of manufactured parts 
and gages arc of ferrous materials having similar cocl- 
ficicnts of expansion, many industries, particularly 
those where tolerances arc in lens ol thousandths of 
inches, have successfully functioned without concern 
ovei the effect of mean environmental temperature on 
manufacturing accuracy In many such situations, an 
arbitrary insistence on 68° F (20° C) temperature con- 
trol leads to unjustifie ] increased cost of manufacture. 

As tolerances become tighter, as the parts become 
bigger, and as the materials of parts and masters be- 
come more dissimilar, the consequences of mean en- 
vironmental temperatures other than 68° F (20° C) 
become correspondingly greater. Here it is to be noted 
that in recognition of the possible consequences of 
mean tvir on menial temperatures other than 68° F 

(20° C), it is not uncommon tc find the following ac- 
tions in use 

(a) Special gaging or masters made of non. mails 
ihc same material as the parts, 

(bj Computation of corrections which arc applied 
to the indicated values of length. The required com- 
putation method is derived from Equation 23 The 
correction is set equal to the negative of the Nominal 
Differential Expansion 

Correction = -NDE t?1) 

Corrected Length = As-read Length + Correction (2?) 


^ t 


:o 
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As I he working tolerance decreases, both of these 
procedures tail to be satisfactory because o! (he mag- 
nitude ol the Uncertainty of Nomina) Differential Ex- 
pansion (see 20.2). 

20.1.2 Measurements Other than Length. Proce- 
dures and formulae for the assessment of the effects 
of mean environmental temperatures other than 68° F 
(20° C) as simple and straightforward as those pre- 
sented in the preceding paragraph are not usually pos- 
sible in cases other than length measurements. 

For example, consider the case of an iron bedway 
casting of a machine. Because the casting may have 
both thick- and rJlin-walied sections, the physical com- 
position of the material may not be homogeneous, re- 
sulting in a non-uniform coefficient of thermal ex- 
pansign . The magnitude of such a variation m expan- 
sion coefficient may be as much as 5 percent. If the 
non-uniformity is distributed as a vertical gradient, 
raising or lowering the mean temperature will result in 
a bending like that produced by a vertical temperature 
gradient. 

This effect is the same as that observed in the well- 
known bimetal strip, and can be called a “bimetal ef- 
fect". 

The bimetal effect in structures of nominally one 
material is relatively small compared with the effect 
of temperature gradients For example, a base casting 
like that mentioned above would have to be subjected 
to a temperature offset of 20° F before the bending 
approaches that induced by an upper and lower sur- 
face temperature difference of only 1° F. However, m 
structures composed of two o r more greatly dissimilar 
materials that are assembled at 68° F (20 w C), the bi- 
metal effect can be quite significant, in such cases the 
effect of mean temperatures other than 68° F (20° C) 
can be properly estimated only by taking into account 
the thermal stresses that exist. 

Existence of severe bimetal effect can be avoided 
only by strict control at 68° F. 

Evaluation of the effects of mean temperatures 
other than 68° F requires that the net effect of the dis- 
tortions of both master and part be determined. 

20.2 Contequences of Uncertainties of Computations 

There are two kinds of systematic errors that occur 
when the effects of mean temperatures other than 
68° F (20° C)are computed. They are the errors in the 
values of the temperatures and in the coefficients of 
thermal expansion that are used in the computations 

Values of temperatures used in computations car, 
be in error because of defects in the instruments used 


in making the measurements or because ol the location 
at which the measurement is made ! m example, the 
thermometer used may U* inaccurately calibrated or 
have a built-in source ol error such as the sell -lical mg 
effect found mrcststancc-hulb thermometers Because 
of the self-heating effect, rcsistance-hulh thermometers 
can be very precisely calibrated in liquid baths and 
give erroneous reading? on metal surfaces or m air be- 
cause the heat transfer process is quite different m the 
different cases. 

Location of the temperature measuring probe is of 
significance because of the possible gradients. Use of 
room air temperature values may introduce errors of 
a degree or more. Readings of direct -con tact probes 
are more reliable but are still subject to error because 
of gradients within the object whose temperature is 
being measured. An effective means of assessing the 
validity of a given location is to compare effects of 
several locations. 

The approach take"' in formulating the standard 
procedure for estimating the effects of Uncertainly n! 
Nominal Differential Expansion is to requite thai 
part and master temperatures be measured to deter- 
mine worst-case deviations from 68° F This procc 
dure, as noted in 6.2.1, for length measurements, 
takes into account the effects of gradients in the ap- 
paratus, as well as in the room tn which it is located 

If part and master temperatures are not measured, 
the estimation of the consequences of uncertainties of 
computations must include consideration of the un- 
certainties in the temperatures used m computing the 
estimation of the effects of temperatures other than 
68° F. Equation 7 is modified as follcv's 

UNE = (5 +0)(L)(t - 68)/!0O7r (23) 

where 6 = Af/f - 68 X 100, or the possible percentage 
enor in the estimated difference between the part or 
master and 68° F. A/ is the estimated possible error in 
temperature difference 

With proper attention to the simple, well-estab- 
lished rules of precision thermometry, the uncertain- 
ties due to temperature measurement can be easily 
reduced. In the usual case, however, the effects of un- 
certainties of coefficient of thermal expansion values 
are much more difficult to overcome 

Coefficient of thermal expansion data are published 
in tables in many handbooks and other sources These 
values cannot be used without consideration of their 
applicability, i.e., their uncertauity. Uncertainties in 
the published data anse because 

(a) The material of the elements of the measure 
ment system -part or master or both -differ from the 
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material for which the data are given. The differences 
may be in chemical composition, physical composi- 
tion, or both. 

(b) The published values are usually the result of 
averaging data from several experiments and from 
several experimenters. Consequently, the data reflect 
the effect of experimental bias. 

(c) The published values are valid only for tem- 
peratures other than 68° F or for a range of tempera- 
ture other than that of the computation. 

The National Bureau of Standards, in calibrating 
steel gage blocks, assumes an uncertainty of the coef- 
ficients of expansion of ±5 percent when the heat and 
mechanical trearment of the steel is known. The 
precision of the coefficient is (1) about ±3 percent 
among many heats of steel of nominally the same 
chemical content, (2) about ±10 percent among 
several heat treatments of the same steel, and (3) about 
±2 percent among samples cut from different loca- 
tions in a large part of steel that has been fully an- 
nealed. Hot or cold rolling will cause a difference of 
about ±5 percent. 

Other materials have their own susceptibility to un- 
certainty of coefficient of thermal expansion, depend- 
ing on the effects of chemical contaminant or physical 
structure. Some materials have gram structure effects 
in terms of expansion coefficients that vary with 
direction. 

The typical thermal expansion measuiement 
conducted with an apparatus called a adatometer in 
which a specimen, usually rod shaped, is heated and its 
change of length measured. Another form of dila- 
tometer measures change of volume by Archimedes’ 
principle, resulting m a coefficient of cubical expan- 
sion. For homogeneous (nondirectionaiiy sensitive) 
materials, the coefficient of cubical expansion has a 
value three Gmes that of the coefficient of Linear ex- 
pansion 

Th. f?n jiat the typical test specimen bean little 
resemblance to real parts, with consequent uncert?in- 
ties in composition and treatment not reflected in ex- 
oenmcntal data scatter, suggests that decreased un- 
certainties can be obtained by direct measurement cf 
each specific object, or full-scaJe dilatomet^y 

Figures 4 and 5 represent two possible ways one 
may find thermal expansion data presented in the 
literature. Figure 4 is a synthetic case deliberately 
oversimplified for the purposes of this discussion 
Figure 5 is ;in actual case * Note that Figure 4 is a plot 


4 Dm courtesv of Rirhftrd K Kirby . U _S N B.S . » i. t x 
p*niion Libor* f or\ 


ANSi 889 j6. 2—1973 



temperature 

FIG 4 Synthetic Experiment*! Results of Thermal Expan 
non Measurements 



FIG S 


of change of length. AA, 3" a function of temperature 
where .1A is defined as zero when :hc temperature 
68° F This is the usual form of raw data from diia- 
tomer experiments 

Figure 5 on the other hand is a plot of the mean 
(or average) coefficient of expansion irom 20° C, 


plotted at t . The data for t - 20° C are derived from 
the slope of the thermal expansion, a A L/dt . at tha t 
special temperature 

Figure 5 gives results from several investigators 
Figure 4 shows how two investigators may obtain dif- 
fering results that are reflected in Figure 5 Both 
Figures show (1 ) the scatter of experimental data an 1 
(2) the nonlinear nature of expansion relative to tern 
perature Data of this type are the source ot all 
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tabulated coefficient of expansion data. The pub 
lathed value, however, varies according to how the ex- 
perimental data are interpreted. For a single investiga 
*.on, the value depends en how the trend is inter- 
preted, i.e., how the average curve is fitted. For mul- 
tiple investigations, the value depends on how the 
data is averaged. 

For example, published values for pure or element 
aluminum is reported as 23.6/° C at 20° C in Metals 
Handbook and 22.4/° C at 20° C in Machinery’s Hand- 
book.* Also, in Metals Reference Book, Table 2, the 
average from 20° C to 100° C is given as 23.9/° C; and 
in Metals Reference Book, Table 1, the average from 
C° C to 100° C is given as 23.5/° C. 

20.3 Estimation of the Consequences of Temperature 
Variation 

An estimation of the consequences of temperature 
variation can very seldom be obtained by direct cal- 
culation Therefore, the procedures described in this 
section are based on an experimental approach lo the 
estimation. 

The basic experimental procedure used in the esti- 
mation of the consequences of temperature variation 
is the drift test which is described in 20.3.1 . Drift test 
results can t>c interpreted in a variety of ways to ob 
tain an estimation of Temperature Vanation Error. 
One n.- ihod is described more fully in 20.3.2 along 
with other methods of interpreting drift test results 
that are not standard, but may be useful because they 
are less conservative and may provide development of 
concrete grounds for negotiating the acceptability of 
thermal effects errors in special cases. 

The rationale for both the dnft test and the esti- 
mation of Temperature Variation Error is given in 

20.3.3 in an explanation of the concept cf the 
3-element system. 

20.2 1 Drift Test Procedure 

20.3.1.1 Equipment. The object of a dnft test is 
to record relative displacement in a 2-element system 
(sec Section 20.3.3). The most direct method utilizes 
electronic utdicaiors whose output is recorded on a 
strip-chart recorder Some measurement processes, 
such as the measurement of flatness with an optical 
flat and monochromatic light or an indicating mi- 
crometer do not lend themselves to the use of auto- 
matic reco ding Therefore, in some cases it will be 
necessary for a human operator to observe thr drift 
and record numerical values and corresponding clock 
Umt. These data can be subsequently hand plotted. 

•Calculated fmm 1 2.44 uinfinT F at 68° F 
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It is strongly urged, however, that wherever pos- 
sible sensitive electronic indicators and strip-char t 
recorders be used. 

Though a drift test can oe performed without any 
necessity for knowledge of temperature variation, it is 
often advisable to record one or more temperatures 
either for use ^ liter correlation of two drift tests or 
for reference if temperature variation is to be later ac- 
cepted as a method of monitoring the process for 
validation of the Temperature Variation Error esti- 
mate. 

Just as in the case of displacement measurements, 
it is strongly urged that all temperatures be automati- 
cally recorded. For this purpose, recording resistance 
element thermometers, especially those wuh ther- 
nvitor sensors, are recommended 

203.1.2 Equipment Testing 

20. 3.1. 2 A Displacement Transducers. Aside from 
the usual calibration checks, electronic indicators 
should be checked for possible sensitivity to the 
thermal environment in which the drift test is to be 
performed. An “electronics drift check'’ should be 
conducted by blocking the transducer <md record, ;ig 
the output for at least the same period of time as that 
of the drift check to be performed. “Blocking" a 
transducer is to make it effectively indicate on its own 
frame, base, or cartridge. Figure 7 shows a cartnogc- 
type linear variable differential transformer billed 
by means of a cap or capture device which holds the 
indicator armature in a fixed position relative to the 
cartridge. 

During the electronics dnft check, the entire dis- 
placement recording system should be located as 
nearly as possible as it wdl be during the drift test 

Electronics dnft tests have been useful in proving 
that, m many cases where electronic indicators have 
been the suspected source of drift, they were innocent 
and the real cause was thermal dnH The commercially 
available cartndge-type LVDT gage heads have been 
proven many times to be especially free from drift 

20.3.1.2.2 Temperature Recording Systems. The 
temperature-measuring and recording apparatus should 
be thoroughly tested for calibration, response, and 
drift. 

Resolution of at least 0.1° F is recommended Time 
constants of sensing elements of about 3 minutes are 
recommended fo* air temperature sensors 30 seconds 
for liquid and surface temperature sensors. Air probes 
must be 'hieldcd from possible radiation effects 

20.3 1.3 Preparauon of System for Test An es- 
sential feature of the feature of the drift test is that 
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conditions dunng the test must duplicate the normal 
conditions for the process as closely as possible. 
Therefore, before the test is started, normal conditions 
must be determined. The step-by-step procedure fol- 
lowed in the subject process must be followed m the 
same sequence and with the same timing in the drift 
test. This is especially important in terms of the ac- 
tions of hum^n operators in mastering and all 
preliminary setup steps. With as iittie deviation from 
norma! procedure as porsible, the displacement trans- 
ducers should be introduced between the part (or 
master, dr ^ending on the type of drift check) and the 
rest of aie C-frame such that it measures relative dis- 
placement along the line of action of the subject meas- 
urement process. 

The temperature-sensing pickup must be placed so 
as to measure a temperature which iscorrelatable with 
the drift Some trial and error may be necessary In 
the extreme case, temperature pickups may have to he 
placed to measure the temperatures of aiNif the active 
elements of the measurement loop. 

20.3.1.4 Representative Time Period for a Drift 
Test. Once set up, the drift test should be allowed to 
continue as long as possible, with a minimum of 
deviation from normal operating conditions. In situa- 
tions where a set pattern of activity is observed, its 
duration should be over some period of time dunng 
which most events are repeated. When a 7-day work 
week is observed in the area, and each day is much 
like any other, a 24-hour duration is recommended. If 
a 5-day -.vork week is observed, then either a full 
week cycle should be used or checks performed during 
the first and last days of the week 

20.3.1.5 Postcheck Procedure. After the drift tes:, 
the displacement transducers and the temperature 
recording apparatus should be rcstandardized. 

20.3. 1.6 Example Drift Test Results. Figures 8 and 
9 are results from drift tests conducted on a measuring 
machine/gage. Figure 8 ts the drift recorded over a 
24-hour period for a system consisting of the master 
and comparator. Figure 9 is the drift recorded over the 
succeeding 24-hour period for a system consisting of 
the part to be measured and the comparator In both 
cases, ambient temperature at a point near the gage 
was recorded and is plotted in the corresponding 
figures. 

20.3.2 Temperature Variation Error. Figure 10 
shows the results of both part/comparator and master/ 
comparator drift tests for a real measurement process 
In this case, ambien. temperature readings were oh 
tamed simultaneously with each drift test for the pur- 
pose of approximating the propei phase relationship 
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FIG 10 Results of part /comparator and mastar /comparator drift tests of a gage m an Inspection Shop Tests werf 
conducted on successive days The data are superimposed on clock time 


The two sets of data were superimposed on a basis of 
clock time, which appears to give a good overall agree- 
ment in ambieni temperature variation 

The ambient temperature variation on the two suc- 
cessive days has a well-defined 24-hour component 
with an amplitude of about 1.5 F. Superimposed on 
this are higher frequency components with periods of 
from l A to 1 Vi hours. From these data it is possible to 
compare the 24-hour cycle characteristics because of 
the repeatability o. ^ie environment at this frequency , 
but phase relationships at the higher frequencies arc 
not discernible because of n^nrepeatability. 

At the 24-hour frequency, the master/comparator 
and part/comparator drift curves are in phase and 
have very nearly the same amplitude. This is a classic 
example that shows the importance of measuring 
cycle time because the larger amplitudes of drift are 
associated with the low frequency , whereas the smaller 
amplitudes of drift are associated with the higher 
frequencies. 

For short measurement cycle times, say 1 hour, the 
procedure for evaluating Temperature Variation Error 
given in Section 20 3 1 results in a TVE = 60min For 
measurement cycle times of 12 hours or more the 
TVE = 120m in 


When the quality of the drift data permits, it is 
sometimes possible to apply the more precise evalua- 
tion methods discussed in Section 20.3.3 which are 
less conservative. In the example of Figure 10, little is 
gained by this procedure because the maximum dif- 
ference between the two drift curves, which corre- 
sponds to the possible error for short measurement 
cycle times is still about 60 microinches. This is 
probably because of nonrepeatafcie components of 
temperature variation in the two days testing The da> 
on which the mastcr/comparalur drift test was per- 
formed appears to have had more severe high 
frequency temperature components This d’serepan^v 
appears to exaggerate the true part/mastcr relative 
drift. Further drift tests to obtain results for more 
consistent temperature variations would be advisable 
m this case if Temperature Variation Error is the major 
thermal effect in this measurement process. 

20.3.3 The 3-Elemem System Concept The mag- 
nitudes of the effects of temperature vanation are de- 
pendent on the structure of the measurement ap- 
paratus and not only on the size and composition of 
the part and master as was true in the previous sec- 
tions. Also unlike the other components of thermal 
error, Temperature Variation Error depends on the 
work habits of the person making the measurement 
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One of tne amplest structure? is that encountered 
in the measurement of the length of an object with a 
gage block and a column comparator. Figure 1 1 shows 
a schematic representative of such a system. As can be 
seen, it consists of a part, a master (the gage block) 
and a comparator. Thus, the system consists of three 
elements. 

In Figure II, each element is shown to have a 
characteristic length; P - part length, M = master 
length, and C = comparator length. In the measure- 
ment process, C is first set equal to M y then P is 
checked to see ifF = C 

If there were no temperature variations, the meas- 
urement process could be straightforward. However, 
because of temperature variations, heat is constantly 
being exchanged between the three elements and the 
ever-changing environment. 

If the time constants of ail three elements are not 
the same, they will respond to temperature variations 
such that it would be possible that all three elements 
will never simultaneously have the same temperature 
And even if the tune constants were all the-»me, and 
their temperatures always equal, they may not have 
the same length, except when all are at 68° F (20° C)> 
because oi different coefficients of expansion. 

For each element, its time constant, length and co- 
efficient of expansion defines its dimensional response 


to temperature variation. 

Figure 12 shows the dimensional response of the 
three elements of Figure ! \ for an assumed sinusoidal 
ambient temperature variation. For simplicity, :he 
hypothetical system consists of three elements of the 
same material but different time constants, .he largest 
being that of the master, the smallest being that of the 
part with the time constant of the comparator between 
those of the other elements. 

As can be seen, the result is that the 3-dimensional 
responses differ in amplitude and phase. It should be 
noted that dimensional response data in this form is 
rarely obtainable, because it requires the use of an in- 
dependent apparatus that must itself be unaffected by 
temperature variation. 

The data of Figure 12, if it were obtainable, can 
easily be interpreted for an estimate of the Tempera- 
ture Variation Error. It is only necessary to consider 
the effec* of the measurement cycle as follows. 

Suppose that at time T m , the comparator is 
mastered. The act of making C and M equal causes the 
dimensional response curve of the comparator to be 
shifted parallel Jo itself (the comparator is "zero 
shifted”) as shown by the dashed curve, if the part is 
checked without delay after mastering, it is found to 
be too large by the amount q. If, instead, the part is 
checked much later, say at time T m2 , the part will be 
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FIG. 12 Sample steady-state dimensional response of a 3 
element system to a sinusoidal ambient-tempera- 
turt variation 


found to be too small by the amount r. If the com- 
parator is remastered at time T ml , the comparator 
curve is again shifted, resulting in new magnitudes of 
possible error. 

Because Temperature Variation Error results from 
the variation of the differences of the characteristic 
lengths, it is possible to separate the 3-eiement system 
into two 2-element subsystems. For example. Figure 
13 shows the two curves that result when the com- 
parator variations (c) are subtracted from the part and 
master dimensional responses (P-C and M-C). These 
data might have been obtained by recording the out- 
put of an electronic indicator, such as is found used 
on modern column comparators, when the part and 
master are, successively, in the comparator w:ih the 
indicator contacting the part and master, respectively. 
Data such as this are the result of drif: tests. !n the 
next section detailed procedures will be given for the 
conduct of drift tests followed by a discussion of 
methods of interpreting drift test data to obtain an 
estimate of Temperature Variation Error. 

The main problem in interpreting such data results 
from the fact that it is not possible to conduct simul- 
taneous part comparator and master /com para tor drift 
tests. Consequently, additional data is required to 
determine the proper phase relationship between the 
two recorded drift curves, or the possible con- 
sequences of unknown phase relationship must be 


considered in the estimalc of Temperature Variation 
Error 

Because the data of Figure 13 have been con- 
structed from the dst* of Figure !2, no phese uncer- 
tainty exists and the Temperature Variation Error can 
be extracted easily. For example, for a mastering 
cycle occurring between times T mi and T ml and a 
meaairement of the part without delay, the possible 
enor q is a^iply the difference between the two 
curves. 

The effect of mastering is to establish a new base- 
line for the part/comparator drift curve (, P-C ). This 
new baseline is shown in Figure 13 as the line (O-O) 
If the measurement of the part takes place at time 
T m * the resultant error is a as determined previously 

If a series of like parts are inspected between times 
T ml and T m v. the consequences of Temperature 
Variation Error range from +(/ to r. 

in the case that the clock times at which mastering 
occurs are unknown and unpredictable and the meas- 
urement cycle time is very short (mastering with each 
measurement and negligible delay before the part is 
inspected), the possible Temperature Variation Error 
is tx, or the maximum difference i ;tween the two 
drift curves at any given tune. Because of the short 



FIG 13 Relative drift components for a 3-element system 
(tami example as m Figure 11). P-0 >s the relative 
drift between the part and the comparator, and 
M-C is the relative drift b e t w e en the master ard 
the comparator 
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FIG. 14 Schematic of artup used to miMurf part on a 9*99 
mtn • Nad-teitw master. 



FIG. 15 Schematic of aatup utad to measure part on a gage 
with a lead-ecrew master The measuring sequence 
it a change from (a) to (b> . 


measuring cycle time the comparator is slaved to the 
master so that the comparator contributes nothing to 
the error. The error, therefore, is 

(PC) - (A/-C) = P~M. 

This error is dependent only on the difference between 
the master/comparator drift and the part/comparator 
drift and the clock tune at which the measurement is 
made If the measurement cycle time is longer than 
the penod of the temperature oscillation, the maxi- 
mum possible error is ± v, or the maximum difference 
between the two drift curves regardless of time. 

Note that v is slightly larger than x. 

in some dimensional measurement processes the 
3-element system reduces to a 2-element system. For 
example, the process of measuring flatness with an 
optical flat under monochromatic light, is a case of a 
2-element system. The comparator here is the human 
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eye which is assumed to have no effective dimensional 
response to ambient tcmpe:iturc variation. 

In length-measuring processes, however, a 3-clemenl 
system is always found. For example, consider the 
cases shown in Figures 14 and 15. The former is the 
case of a measuring machine or machine tool with a 
ieadscrew serving as master. In Figure 15, .he meas- 
urement process is shown to consist of changing from 
position (a) to position (b). The analogy between this 
case and the simple 3-element system of Figure 1 1 is 
seen if it is realized that in the two configurations, the 
comparator is composed of a portion of the lead- 
screw, the nut and table support for the part. These 
elements, though appearing to change, remain in a 
structural loop, while the part and master exchange 
places as members of the loop. 

The case shown in Figure 16 is that of a I -inch 
indicating micrometer used as a comparator. The 
master is a gage block. Figure 17 shows the same 
micrometer used to measure the part without check- 
ing zero. In this case, the micrometer frame plus 
screw, opened to the size of the part, constitutes the 
master. The same structure also fulfills the function 
of comparator. 

In Figure IS still the same rr rometcr is oroughi 
to its null position and a zero correction is made be 
fore the part is measured. In this case the master is 
that portion of the screw that is withdrawn to make 
room for the part. The rest of the micrometer forms 
the :omparator. 

Consider now a 2-inch Indicating micrometer and 
the following case. The part is \ Vi inches in diameter. 




FIG 16 
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FIG 17 

A I -inch gage block is used to master the micrometer 
The master in this case is the gage block plus that 
portion of the screw, approximately &-inch iong, 
which is withdrawn to make room for the part (see 
Figure 19). 


The last four cases show how the master and com- 
parator can be changed by changes »n tin upiCitfiiug 
procedure. 

20.4 Coraaquancas of Tampa rami ra G radiants 

Effects of gradients in mean environmental temper- 
ature are usually accounted for in the case of length 
measurements under effects of temperatures other 
than 68° F or under considerations of uncertainties of 
temperature measurements. Consequently, the mam 
concern here is the effect on measurements other than 
length such as a measurement of flatness. An example 
of the estimation of the effect of temperature gradient 
on a measurement of flatness is given in Section 10 1 

To satisfy the intent of the Thermal Error Index, 
computation of an estimate of the consequences of 
uncertainties of computations as discussed in Section 
20.2 must be performed and added to the estimation 
of the consequences of temperature gradient and the 
consequences of temperature variation (Section 20 M 

20.5 Thermal Error Index 

This standard docs no* recommend values lor the 
Thermal Error Index Such values cannot be stated 
without regard to other sources of erroi in the meas- 
urement process. For example, a Thermal Error Index 
of 10 percent assigns to thermal effects that fraction 
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of the working tolerance that is usually considered to 
include the composite effects of alt error sources. In 
given cases, the permissible values depend on the 
degree of control that is maintained ove' al) aspects of 
the measurement process, including th skill level of 
personnel. In a machine shop, a value of 0.1 may be 
justifiable while in a metrology laboratory it may be 
possible to increase the value to 0.2. 

TEic main objective of the Thermal Error Index is 
to convey the quality of a measurement process with 
respect to thermal effect. As such, it is mainly an ad- 
ministrative tool. 

It is to be noted that one way to reduce a Thermal 
Error Index is to increase the working tolerance. Con- 
sequently, it serves as a feedback device tc inform 
management and designers of the degree of absurdity 
of a specified tolerance. The Thermal Error Index does 
nothing more than estimate the maximum possible 
error caused by thermal environment conditions af- 
fecting a particular measurement process. It doe* not 
establish the true magnitude of error in any measure- 
ment It serves to remove doubt about the existence 
of errors and to establish a system of rewards and 
penalties to processes that are combinations of tech- 


niques and conditions, some good and some bad. 

A Thermal Error Index evaluation penalizes a 
measurement process on three counts. 

(a) Existence of temperatures other than 68° F 

(b) Existence of temperature variations 

(c) Existence of temperature gradients 

The same evaluation rewards good techniques by 
decreasing the Thermal Error Index for (1) attempt- 
ing a correction foi the consequence of temperatures 
other than 68° F,(2) keeping environment^ variations 
to a minimum, and (3) maintaining acceptable tem- 
perature gradients. The act of performing *he evalua- 
tion results in the knowledge of what techniques or 
conditions can be changed to achieve the greatest im- 
provement with the least effort. For example, if Tem- 
perature Variation Error is found to be the greatest 
source of error, the measurement cycle time may be 
reduced such that the Thermal Error Index is reduced 
to an acceptable value. Thus, by more frequent master- 
ing, at some nominal increase in operating expense, 
possible misapplication of capital to improve tern 
perature control is avoided. 
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9.14 ERROR BODGETS 


Jl abort R. Donaldson, Project Leader 
Large Optics Diamond Turning Machine Project 
Lurrence Livermore National Laboratory, Livermore, California 


INTRODUCTION 

Generally speaking, an error budget is a systems analysis tool, used for 
prediction and oontrol of the total error of a system at the design stage for 
systems vbere accuracy is an important measure of performance. Given a 
system-error goal, an error budget can be used in a oontrol mode to set 
individual subsystem error limits, while also making trsde-offs that balance 
the level of difficulty amon? the subsystems. In a predictive mode, proposed 
subsystem design can be assessed for error contributions, leading to a 
predicted overall system error, lypically the predictive and oontrol modes 
are iterated repeatedly. 

A significant break-point occurs when the required subsystem error limits 
exceed the state of the art. Within the state of the art, the error budget 
can be used as an aid in minimizing ths overall system aost. Beyond the state 
of the art, the trade-off alternatives are those of accepting a less stringent 
system-error goal or accepting the time and oost required for the necessary 
improvements in the state of the art. Obviously the larger the required 
improvements the more speculative the development process becomes, but the 
error budget remains useful as an assessment tool. 

To the author's knowledge, error budgets have not been used in the design 
of machine tools beyond the example to be presented later in this article, and 
hence the subject is not well developed. The following section contains the 
general approach applied to the example; suggested areas for further 
deye-lypmer.>*3$.e ^iven at the end of the article. 


MACHINE TOOL ERROR BUDGETS 


BASIC ASSUMPTIONS 

Two basic assianptians underlie the use of an error budget for a machine 
tool. The first is that the instantaneous value of the total error in a 
specified direction is tne svm of all the individual error components in that 
direction (i.e., linear superposition is valid). The second is that the 
individual error components have physical causes that can be isolated and 
controlled or measured to allow reduction or prediction of the error 
magnitude. Both assumptions are supported by the discipline of machine tool 
metrology, which provides methods for the measurement of individual errors, as 
discussed in Sec. 7.0. 

ERROR BUDGET FIOK CHART 

Figure 9.14-1 shows ♦'he overall process of generating an error budget, 
illustrated for a two-axis lathe, from the physical sources of error at the 
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left to ttfcr error* of the workpiece et tbs right. The following paragraphs 
will elaborate an the various stages; underlined tens are defined in the 
sentences in which they first appear. 

BOOR SOURCES, COUPLING MECHANISMS AMD DXSPIACZMEMT ERRORS 

The dlsplacenent errors are the distances between the surfaces of an 
actual and an ideal (perfect) workpiece. The dlsplacenent errors are always 
Measured in tbs sensitive direction , i.e. , normal to the ideal workpiece 
surface. It is useful to note that even for a general-purpose machine tool 
intended far many part shapes, the sensitive directions may be restricted — 
e.g., for i lathe, the X-Z (slide axis) plane containing tbe spindle 
centerline is tbe plane of all sensitive directions during cutting, and hence 
comparable displacement errors in tbs T direction are n onsensitive , being 
tangent to tbs workpiece surface. 

An error source is tbs physical causa of a displacement error, and a 
coupling mechanic is the physical factor that connects the two. Error 
sources need only be as fundamental as is useful, i.%., surface finish can be 
degraded by seismic ground motions which can be further explained by study of 
the earth sciences, but acme knowledge of tbe amplitude and freque..-/ of the 
seismic motions is sufficient. In this example, reducing the resulting 
displacement error to an acceptable level depends entirely on tbs design or 
••lection of adequate vibration isolators for tbs eoupling mechanism. This 
example also Illustrates tnat the coupling mechanism is in general a transfer 
function in tbs dynamic system sense. Tbs transfer function *-y range from 
elementary to very implex. For example, el i deway positioning error due to 
Abbe offset (perpendicular distance frem tbs cutting tool to tbe line of 
action of tba position measuring system) has angular motion of the slideway as 
the error source and the offset dlstanae as tbe coupling mechanise, making the 
transfer functioi a simple multiplier. At tbe other extrase, room- temperature 
variations as an. error sourot have a ocsiplicated time-history effect on 
tool- to-wor)^i tee displacement error3. 

There are two difficult points in dealing with the error course i in an 
error-budget analysis. The first is generating a complete list that do*' not. 
roit any aignificant sources. Here the past experience of those ocopiling the 
list, is the primary factor; there is no known test for complete *.o*s. The 
second area of difficulty is in assessing magnitudes, both for the error 
source and for tie displacement error resulting from th« r^'rce and the 
coupling mechanism. Ideally one would prefer to have complete detail on the 
displacement errors E i? including their variation within the machine work 
rone and with time. However, it is unreasonable at the design stage to expect 
more than a peak- to- valley envelope bound on Ej, together with some 
approximate knowledge of the frequency of variation (temporal or spatial,' 
within the envelope. Magnitude estimates should be supported by calculations 
wherever possible: these may range frem simple or der-of -magnitude estimates 
ie.g., temperature stability of 0.01°C means a steel bar (a - 11.5 x 10 -t 
per *C) of 1 ■ length will change in length by 0.115 in, to computer-based 
finite-element calculations of structural deformations uncJer load. Sane 
magnitudes can only be estimated, such as the deformation of the workpiece by 
the clamping fixtures or the wear of the cutting tool. 

It is worth noting that the machine tool designer is usually faced with a 
problem of divided responsibility in oonstructing *n error budget, unless a 
ccmplete turnkey facility is being provided for machining specified 
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workplaces. If tbs user locate* the machine in s poorly controlled thermal 
environment, or if the atduie tuul it pwpcss« s!m h «»wi to be used 

with a wide variety of wori^iec* shaper, water icu.s, fixtures, and cutting 
tools, the error estimation process beoanes wore difficult. Nevertheless, it 
is trecoea ended that all error sources be included that influence the final 
workpiece accuracy. If desired, the final result can be recomputed with 
various sumptions regarding those error uouroes having divided responsi- 
bility, to determine the sensitivity of the totai error to these assumptions. 

WORKPIECE ERROR CATEGORIES 

Considering a circular cylinder as a simple but specific example of a 
workpiece, it is clear that there can be several workpiece error categories , 
such as: 

e Size (di Meter „ length). 

e Fora (roundness, straightness, end flatness). 

• Surface Finish (side or end, with or across lay). 

Since different error sources 5^ say produce displacement errors that 
fall into different workpiec* error categories, A, B or C, in Fig. 9.14-1, it 
is important to choose categories that are meaningful for the intended end use 
of the workpieces '* possible (again raising the problem of divided 
responsibility), tfc.ee that the categories given can be separated according to 
the spatial frequency of variation along the work, lece surface, with average 
size error being a "<ic" (zero frequency) term, sur.-ce finish having the acst 
rapidly varying spatial frequency and form arrors being in between. When 
dealing with error sources having a temporal frequency, it is necessary to 
determine the spatial frequency that will result on the workpiece surface by 
use of typical machining parameters. For example, a lathe spindle turning at 
1200 rpm (20 cps ) «ith a feedrate of 0.1S aa/rev will yield a feed speed of 
3 m/s. If surface finish is considered to consist of wavelengths shorter 
than 0.75 na along the workpiece surface, then the temporal frequency in the 
surf ace- finish category will be higher than (3 m/sec)/(0.75 am) , or 4 Hz; 
lower frequencies will contribute to form errors. Increasing (ct decreasing) 
the spindle speed or feedrate per revolution will increase (or decrease) the 
transition frequency of 4 Bz. Since any given source S j may cause a 
displacement error in more than one workpiece error category, each 
possibility should be examined. 

ERROR DIRECTIONS 

After qmeration of a list of uisplacement errors in each category, it is 
preferable to separate these < nto error directions along the machine tool axes 
(X and Z for a lathe. Fig. 9.14-1). In the surface-finish category, this may 
not be feasible, since many of the sources are vibrational, and it is 
difficult to predict the direction of vibration in a complex machine tool 
structure. The reccomended method is to estimate a maximum amplitude and 
assume it to be am~idirectianal. 

COMBINATORIAL RULE 

Given the displacement errors by category and direction, the next 
question is what combinatorial rule to t e in combining t K em into a single 
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displacement error. If the Mt of errors bed complete detail, direct addition 
could be used to generate a cap of the resultant displacement error as a 
function of slidaway position, tine, etc. In the core probable case of no 
detail asespt rs upper bound for the magnitude of each error, the selection of 
a ooebinatorial rule is acre difficult. The end use of the workpiece is also 
a factor in the selection process. In soma applications an averaging process 
occurs, making an raa error vplitude meaningful, while in others , the largest 
Isolated error peak la crucial, regardless of tbs perfection of the retaining 
surface. In the first case a statistical treatment is obviously Indicated. 

In the latter extreme, a conservative approach it to sai the individual 
displacement-error implitudes arithmetically. However, this approach aay be 
extremely conservative, due to the very low probability of all N errors being 
at a maxima simultaneously; statistics can also aid in assessing the 
probability of exceeding some submaxiatm aplitude. 

The ras error amplitude RMS tot for H individual errors of varying rms 
amplitudes RMS^ , providing the individual errors are uncorrelated, ia simply 


RMS 


tot 





1/2 


( 1 ) 


Use of Bq. (1) requires an ras amplitude for each displacement error, 
whereas the preceding discussion has used a total or peak-to-valley 
amplitude PV^ (separation of two parallel lines containing the error 
signal) . The two amplitudes are connected by an equation of the fora 


PV t • K • RHS j 


( 2 ) 


where X is a niaerieal factor depending on the probability distribution 
of the error signal between the bounding lines. n>e values of X far 
three dit.tr Lions— a pure sinusoid, a signal with uniform probability 

density, and i2-sigma Gaussian— -are 2.83, 3.46 and 4 0 respectively. 
Unless more detailed information is available, the middle case is 
recommended (X • 3.46). Individual error traces are typically not 
Gaussian, although they aay show seme central tendency, causing the 
uniform density cssumption to be acmewhat conservative; in any case the 
other two values of X differ only by about *20%. The result is then 
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(3) 


Bg. (3) can be applied in Fig. 9.14- to error amplitudes to 

to yield and so on for X A , etc. 

KORKPIUCZ ERROR 


Given the resultant displacement error by direction for each 
workpiece error category, the workpicoe error o. Fig. 9.14-1 can be 
predicted in each category from knowledge of the workpiece geometry , 
i.e., the aise and shape as shown on an engineering drawl, j. For 



workplace surfaces hawing a slope angle 4 with respect to the X axis, the 
focmula far the error normal to tbs surface in category A is 
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and similarly far other categories. Bq. (4) describes an ellipse with major 
and minor values of end E AZ . Xf and Ej^ Are equal, the ellipse is a 
circle and the error is cmni directional. Xf the error is not omnidirectional 
and the worlqpiece contains a continuously varying contour, Bq. (4) can be used 
to calculate tbs error at selected points along tbe contour for the purpose of 
estimating an average error. Fee cases where tbe workpiece geometry is 
unknown, this section cm be omitted. 


ERROR BUDGET FOR TBE LARGE OPTICS DIAMOND TURNING MACHINE 


BACKGROUND 

A properly sharpened natural diamond cutting tool will cut in exact 
accordance with its edge location and can produce e mirror finish on a variety 
of materials if the machine tool has a sufficiantly low vibration level. This 
has created interest in tbe use of machine tools to fabricate reflective metal 
optics. While it offers the possibility of radically aspheric shapes that are 
not reasonable to produce by conventional optical figuring and polishing 
methods, it also requires accuracies that are 10-100 times more stringent than 
those usually expected of precise machine tools. Efforts in this direction 
have progressed to the point that diametars below 0.5 m can be diamond- turnad 
to figura accuracies of the order of 100 la (4 yin.) ms and diameters up to 
2 a done far accuracies of tbe order of 2S0 nm (20 yin.) rms. Interest in 
accuracies down to 13 nm (0.5 yin.) in sizes 19 to 1.5 a led to a study for a 
new machine, known as the Large Optics Diamond Turning Machine. Because the 
achievement of these goals would require a major affort, development of an 
error budget was made a part of & preliminary dasign study. Due to time 
constraints on subsequent design and construction, improvements in the state 
of the art were limited to moderate extensions by methods reasonably 
well-defined at the outset. 

An overall view of the above machine is shown in Fig. 9.14-2. The design 
and the error budget evolved by mutual Interaction; spaoe does not permit a 
full description of the process or of the resulting design. However, 

Fig. 9.14-2 illustrates several features: 

e Ose of symmetry, such as a vertical spindle axis (gravity along 
workpieae centerline) and a bridge design for increased rigidity. 
Symmetry causes several error sources to be self-cancelling (zero 
coupling) . 

e Interferometric tool-path measurement, with interferometer beams 
contained in evacuated telescoping tubes. 

e Metrology frame, a structure kept free of variable loads to serve as a 
measurement referenoe datin. 

Ot.»er features will be noted in dir cussing the various error-budget entries. 

Two workpiece error categories will be presented, surface finish and 
radial figure error. * wavelength of 0.1 mm (0.004 in.) was chosen as the 
transition between the two. For a finishing cut feedrate of 4 ym/rev and 
100 rpm, the associated time Is 15 s. For th- optical systems in question, an 
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7IG. 9.14-2 Artist’s' concept of Large Optics Diamond Turning Machine. 
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ns error amplitude is meaningful and was used in both categories. 

Displacement errors were estimated as peak- to- valley amplitudes and the 
uniform probability density assiaption leading to Bq. (3) was employed. 

SURTACK-PINISH ERROR BODGET 

Surface finish is degraded by a variety of sources, many of which are 
difficult to quantify. ?or this reason the entries in Table 9.14-1 are 
largely derived frem experience and are treated as omnidirectional. 

The first four error sources in Table 9.14*1 are sources of vibration 
excit tion. ficternal mechanical disturbances come from the fact that the 
pneiaatic isolators transmit same ..recti or of seismic disturbances and 
building vibrations to the machine, as do various electric and fluid power 
lines. Room noise is also an excitation source; a machine room with 
sound* insulated walls is required. Bydraulic vibration refers to noise 
created by flow of temperature-controlled liquids through or over the machine 
to maintain a stable temperature field. The spindle drive entry is similar to 
the first category, but refers to excitation from the floor-mounted motor 
being transmitted by a low-influence drive ooupling. Imperfect spindle air- 
pressure regulation will cause the spindle faceplate to rise and fall; the 
entry shown corresponds to 0.03% change in gage pressure for time periods 
shorter than IS s. Theoretical finish is the height of the scallop-shaped 
cuts caused by the finite ad/ance- per revolution of e round-nosed cutting tool 
and is one of the few entries that can be calculated. In fact, the magnitude 
was chosen to be equal to the other atelier entries, leading to the feedrate 
quoted in the earlier calculation of the IS s transition time (assuming a 
0. 75 -m (0.30-in.) tool radius). The estimate far imperfections from the tool 
edge and the cutting mechanics is based on experimental data from the 
literature. Tha resolution of the control system becomes significant when 
approaching tangency to one axis. The servo-controlled tool mount is a 
law- amplitude, high-bandwidth position servo mounted on the lower end of the 
tool bar, driven by the position-loop error signals of the main slideways; the 
entry represents a combination of the tool mount resolution and the accuracy 
of its position measurement wsensors. The last two entries in Table 9.14-1 are 
associated with the interferometers. The first applies to the distance 
interferometers, and is a consequence of imperfect bea»-spli tting of the 
two-frequency laser beam, causing a cyclic displacement error once per 
wavelength (633 ran) . The last applies to the straightness interferometers, 
which operate in still air at 1 atm and are subject to any difference in index 
of refraction caused by pressure or temperature differences between the two 
equal-length measurement arms; the estimate is based on experimental data. 

Performing the root-sum-square operation of Eq. (3) , the ms value is 
found to be 4.2 nm or 42 A (0.17 pin). This result is somewhat smaller than 
is typically observe*! on existing diamond turning machines, especially larger 
machines, but is larger- than the best achievements or. coaller machines, which 
approach 10 A ras; hence the result is achievable in principle. 

The numerical results of Table 9.14-1 can be used to illustrate a feature 
of ms error amplitude when the individual amplitudes are comparable, that the 
total increases aR>roximately as N 1 / 2 (exactly, if all amplitudes are 
equal). Adding another entry of average size 4 nm (0.16 uin.) increases the 
calculated result by about 4% co 44 A, as does multiplying by (12/11) 1//2 
* 1.04. Thus the consequence of overlooking an independent error of a typical 
magnitude is not serious if N is reasonably large. Another feature of ms 
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BfcJBI X 9.14-1 Surface-finish error budget 


Krror aouroe 

Feak-to-Valley 
amplitude 
is (yin.) 

1. 

Btternal mechanical disturbances 

3.0 

(0.20) 

2. 

Airborne noise 

2.5 

(0.10) 

3. 

■ydraulic vibration 

2.5 

(0.10) 

4. 

Spind^ drive 

2.5 

(0.10) 

5. 

Spindle air pressure 

2.5 

(0.10) 

6. 

Theoretical finish 

2.5 

(0.10) 

7. 

Tool-edge/cuiting eechanics 

4.0 

(0.16) 

8 . 

MCD resolution 

5.0 

(0.20) 

9. 

Servo-aontr oiled tool aount 

10.0 

(0.40) 

10. 

Xnterfercseter phase distortion 

2.5 

(0.10) 

11. 

Index difference-straightness 
interferometer 

3.5 

(0.14) 


Sum of squares 

- 215.75 (0.3452) 


res value (Bq. 3) » 4.2 na (0.17 uin.) 

or 42 A ns 
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•rror# , the tendency of the larger entries to dcaainats the results, can also 
be shewn frea Table 9.14*1. Zf entry 9 (10 m) had been caittad, the result 
would drop by about 25% to 31 A; If ntry 9 were Increased to 20 rot, the new 
result would becoae 66 A, over 501 larger. 

RADIAL FI GORE -ERROR BUDGET 

Figure error along a radial path is distinguished froa a-.iauthal figure 
error (along a circular path around the axis) , the latter ueing caused, for 
exaople, by errors in the spindle truth of rotation. Figure errors are 
distinguished frea surface-finish errors by spatial wavelengths longer than 
0.1 oai (0.004 in.) or temporal frequencies lower than 1/15 Bz. 

Table 9.14-2 shows the various sources of radial figure error and the 
associated amplitudes in the X and Z directions, which will be discussed in 
turn. 

The interferometric aeasureaent of position is fundaaentally limited by 
the stability of the laser frequency. The error magnitude corresponds to a 


TABLE 9.14-2 Radial figure error budget. 




Peakao-valey magnitudes 

Error circs 


X4ir action 

am (mw.) 

Z-d traction 
nm Um.) 

foation interferometers 
Ukam central frequency 
lades of refraction 
Optical, electro sec factors 


3.0 (0.12) 
3.0 (0.12) 
5j0 (0J0) 

55 (0J2) 
55 (0J2) 
5.0 (0.20) 

Stnfhtnesi interferometers 
Ffcth length difference 
Optical fkt dffavnee 
Index diffvence 

Raaoiu don— metrology frame drift 


3.7(0.15) 
15-0 (0.60) 
3.5 (0.14) 

3.7 (0.1 S) 
13.0(0.60) 
3-5 (0.14) 
6.3 (0.25) 

Control system 

Servo-con trot led tool mount 


10.0 (0.40) 

10.0(0.40) 

Tempmaeiae control 

Metrology structural loop 
Spindle growth 

Workpiece th«m*J boundary kyer 


10.0 (0.40) 
30.4(1 27) 

75 (0.30) 
40.0 u. 60) 
34 4(1.38) 

Sptndkav supply preuure 


— 

75 (0.30) 

Gnvruttonel loading 


5 J> (0 JO) 

— 

ftwometnc pressure 


4 JO (0.16) 

5.0(0.20) 

Monmachme factors 
Tool nose round ness 
Workpiece fixture distortion 
Workpiece body forces 
Workpiece intvnaf unbalance 
Workpiece re nduaJ ureas 


23.0(0-50) 
75 D (3.00) 
25.0 (1.00) 
J .0(1.00) 
25.0 (1.00) 

1 3.0 (050) 
60.0(2 40) 
20.0 (0.80) 
20.0 (0 30) 
20.0 (0.80) 


Sum of qiarn 

9125 (14.60) 

8207 (13.13) 


rm« viiuee ( tc . 2) 

27.6 am <1.10 mul) 

26.1 run ( 1.05 win.) 
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drift of 5 parts in 10*; this value is vail vittain the stats of the art for 
laboratory lasers using iodine-cell stabilization, but is ziore stringent th 
the 3 parts in 10* far aomercially available heterodyne lasers and hence 

ml Ic^ml uvv«Iu|Ai«iil uc. u cvuuctiun t» ooswer ci ai 
practiaa. The difference in the numer i cal values for X and Z cost from the 
maximum difference in arm lengths, which is mmallar for X even though it has a 
larger travel, because of the symmetric arrangement of beam paths. The 
variation of the index of refraction within the bean path, also 5 parts in 
10* , can be achieved by maintaining a vacuum in the beam tubes between 1 and 
15 millitorr. The optical and electric factors include items such as beam 
collimaticn and spatial filtering, terror scatter and non-flatness , etc., 
which tend to produce poorly defined fringes, plus detector noise, amplitude 
sensitivity and electronic jitter; the amplitudes are estimates. 

Figure 9.34-3 show a schematic diagram of the straightness 
interferometer for the X axis. As noted earlier, the straightness 
interferometers operate in a still-air enclosure with arm lengths that are 
nominally equal; errors can be caused by actual differences in arm length due 
to inaccurate system setup, or by air currents, etc., that cause index 
differences between the two arms. Since the beam* traverse the two optical 
flats as the X-position changes, differences in flatness cause errors. This 
error is very sensitive; the amplitudes given correspond to measurement of the 
flatness differences to about 4 na (1/6 yin.), with storage and subsequent 
recall from a microprocessor memory. The geometry of the arrangement shown in 
Fig. 9.14-3 is also sensitive to slight rotations of the metrology frame; a 
scheme for measuring such rotations using the difference of the upper and 
lower interferometer readings is resolution-limited by the amount shown. 

The only long-term control-system error is the residual error of the 
servo- con trolled tool mount, which as noted earlier is driven by the error 
signals of the main axis drives. The error magnitudes shown are ba^ed on 
resolution and linearity error of the tool mount displacement sensors. 

The metrology structural loon will be temperature stabilized by a large 
flow of water within an enclosing shroud (not shown in Fig. 9.14-2); the 
magnitudes shown are calculated frero an ability to measure and control the 
incoming water temperature to LI x 10~ 3o C (2 x 10“ 3o F) and a possible 
temperature gain or loss of half this amount from inlet to outlet. These two 
error sources may be correlated and hence are assumed to be additive rather 
than independent. The estimated value for spindle growth, due to heat 
generated during operation, is based on extensive computer calculations of an 
internal forced-air cooling system. The workpiece thermal boundary layer 
arises from the fact th*,t most of the energy expended in rotating the 
workpiece in the sea of air around the machine is dissipated in a thin layer 
at the workpiece surface, which becomes significant for the case shown of a 
1.5-ra diameter rotating at 100 rpm. 

The source of error associated with the spindle air simply is the same as 
in the preceding section exoept that the present estimate is for long-term 
drift. Gravitational loading is due to small tilt errors in the self-leveling 
system of the pneumatic isolators, causing a distortion of the metrology frame 
in the X direction. Barometric pressure effects arise because of the 
evacuated beam tubes; the tube area times the extremes of atmospheric pressure 
amount to force variations of a f.w pounds on the metrology structure. Both 
this and the gravitational load displacement were calculated fran 
finite-element models of the proposed design. 

The final category of nonmachine errors represents the area of divided 
responsibility for this example. One is the out-of-roundness of the 
diamond-tool nose; the error amplitude represents an accuracy of measuring the 
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tool nonroundnass and programing the tool path to compensate. The regaining 
errors arlaa fron firturing distortion and other forces associated with 
rotating and cutting an unknown workplace. While the estimates for the 
workpieoe factors are the largest in the error budget, they are also 
considerably mailer than is presently achieved in diamond turning practice. 

Applying Mg. (3) to the X and S error magnitudes yields the values shown 
of about 27 m (1-1 yin.) for both axes. Since tne two values are nearly 
equal, the rs'iltant error is omnidirectional despite the variations of the 
Individual axis components . 

It can be noted that the aost stringent goal of 13 nm (0.3 yin.) rms 
was not achieved. The dominant factors in the error budget are the final 
group of non -machine factors; tha goal oouid be reached in the X direction if 
all factors in the group were reduced to about 10 nm (0.4 yin). However, 
the goal could not be reached in the Z direction even if all of the nonmachine 
factors were xero; further reduction in the spindle growth and the workpiece 
thermal boundary layer (e.g., fay operating at very low spindle rpm) would be 
necessary. 

As a final obsarvation, it should be noted that aeveral error souroes 
were excluded from Table 9.14-2, as shown in Table 9.14-3. The first two are 
axis-alignment errors that are angles rather than displacements and hence 
dependent on the radial width of the workpiece surface of interest. The 
second two can be shewn to cause a curvature error. For optical components 
these errors were judged worthy of special treatment; in a more general case 
they would be included in Table 9.14-2. 


TABLE 9.14-3 Special figure errors. 


Error louroe 


PMk-4o*mtt»y amplitude 

X4xu frames* to wpv*l\t 


0.1 mta4 

Z-M.XU pnUkkim to ip indie 


0.1 MTtd 

Tool ttfUng-X direction 


50 nm (2 uto.) 

Tool not* radio*, axe 


13 nm (0.5 mit.) 


CONCLUSIONS AND RECOMMENDATIONS FOR FUTURE WORK 

An error budget for machine tool accuracy subdivides the overall problem 
into a number of smaller steps, thereby providing a more systematic approach 
as well as a greater degree of scrutiny of the various details. 

The example cited in this paper is clearly esoteric; a valuable step 
would be the development of a checklist of error sources for more co mm on 
levels of accuracy. 

Implicit in the discussion of errors is an assumed ability to measure 
them; hence a given firm interested ir designing to increased accuracy may 
have to begin by improving their capability in machine tool metrology. 

The development presented herein assumes an rms combinational rule as 
suitable. This wo-k could be extended toward applications concerned with the 
warst peak- to- valley error by presenting a method for calculating the 
probability of reaching increasingly larger fractions of the maximum amplitude. 

Another possible extension of the error-budget concept is to aid in the 
writing of specifications between buyer and seller for the acquisition of 
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Machine tools. A typical position taken by tbs buyer is to require that tbs 
*•11 sr provide a aachins tool capable of producing one or note workpieces 
within the tolerances of the workpiece drawing, placing tbs full burder. of 
T'.g. 9 . 14-1 on the seller. The «»ller is sere incliuw Ui wgree to a 
per for* anew specification that wets limits on errors by category and 
direction, nearer the center of Fig. 9.14-1 (positioning accuracy, 
straightness, squareness, etc.). Through an understanding of what accuracy ia 
required for his particular class of workpieces, and application of the 
error-budget approach to the seller's performance specifications, it should be 
possible for the buyer to connect the two views quantitatively. This approach 
s bo aid also allow a rational ooaparieon between the performance specifications 
of competing sellers. 

It should be eqihesised that significant reductions in the requireaents 
for machine tool accuracy car. arise fro* examining the geometry of the class 
of workpieces to be machined. For example, in machining optical components 
(such as parabolic mirrors), two factors are significant. First, the 
sensitivity to errors in the radial (X) direction is proportional to the 
"speed* of the optic, end is an order of magnitude mailer than the axial (Z) 
sensitivity at tha outer adoe of e moderately fast f/2.5 optic. Secondly, a 
number of sources lead to Z errors that vary linearly with X (spind_„ 
squareness to X, Z lead error in positioning, radial toolsetting error (to 
first order), the linear portion of axial spindle growth vita time (if 
constant rpn and feedrate are used)). For optical components, the effect of 
linear error terms can be significantly reduced by selecting another best-fit 
curvature and making a mall compensating adjustment in the axial mounting 
position. Since the ccmercialization of machined optics in the infrared 
region is underway, a study mbit* quantifies tbsse reduced error sensitivities 
far optical workpieces would be valuable. 
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